You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the FSST encoding is only applied when the data is larger than 4MiB. This is an artificial limitation because I was seeing bugs when encoding smaller strings. In addition, FSST is fairly complex, and we need to be highly confident that it will not write corrupt data (writer corruption is difficult to recover from since the data is usually lost).
Let's enhance our confidence in FSST with some additional testing and make sure any threshold in FSST application is meaningful and well understood.
The text was updated successfully, but these errors were encountered:
Currently the FSST encoding is only applied when the data is larger than 4MiB. This is an artificial limitation because I was seeing bugs when encoding smaller strings. In addition, FSST is fairly complex, and we need to be highly confident that it will not write corrupt data (writer corruption is difficult to recover from since the data is usually lost).
Let's enhance our confidence in FSST with some additional testing and make sure any threshold in FSST application is meaningful and well understood.
The text was updated successfully, but these errors were encountered: