-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
get_optimal_index_keys_v2 support faiss AutoTune #139
Comments
Autofaiss is an alternative implementation of faiss AutoTune that works based on the contraints listed in the readme. |
I think the faiss readme is a recommendation rather than a standard, it's a tradeoff between performance and recall. but autofaiss autofaiss/autofaiss/external/optimize.py Lines 174 to 178 in d5c773f
I recommend randomly sampling the embeddings and evaluate recall/perf by various index key parameters to get optimal index key,such as https://github.com/erikbern/ann-benchmarks/blob/8807d6ead4cf14318ac43166cdabf02b491f620e/algos.yaml#L146-L187 |
Feel free to try it out. Beating autofaiss current heuristic on all
relevant metrics (latency, recall and memory) while taking a reasonable
amount of compute (IE not recomputing the whole index many times) would be
great!
…On Thu, Nov 17, 2022, 14:15 xiongqiangcs ***@***.***> wrote:
I think the faiss readme is a recommendation rather than a standard, it's
a tradeoff between performance and recall.
Examples:
http://ann-benchmarks.com/sift-128-euclidean_10_euclidean.html
[image: image]
<https://user-images.githubusercontent.com/7856886/202449023-559804e3-a84d-404e-a7aa-13478543cdec.png>
but autofaiss
https://github.com/criteo/autofaiss/blob/d5c773fa8ab78ae0dddb22cad60832c55eadc999/autofaiss/external/optimize.py#L174-L178
I recommend randomly sampling the embeddings and evaluate recall/perf by
various index key parameters to get optimal index key,such as
https://github.com/erikbern/ann-benchmarks/blob/8807d6ead4cf14318ac43166cdabf02b491f620e/algos.yaml#L146-L187
—
Reply to this email directly, view it on GitHub
<#139 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAR437XMKA2MCZMJTSLX4ELWIYVWJANCNFSM6AAAAAASCAVI3Y>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hello @xiongqiangcs ! We chose to use a heuristic rather than benchmarking n different indices because it is n times faster and that the heuristic is not too far from finding the best index. But indeed, benchmarking n index keys could give better results. There might be good ideas to use to make it possible to benchmark quickly several indices while not taking too much time! We are open to any suggestions if you want to try it out 😁 |
autofaiss/autofaiss/external/optimize.py
Lines 139 to 159 in d5c773f
The text was updated successfully, but these errors were encountered: