zilliztech / bbann Goto Github PK

• add histogram of cluster size
• nprobe as x axis

https://gitee.com/bianzheng123/nips-competition/tree/master/dataset-partition

https://docs.qq.com/doc/DWWVTSUdvZEhudGVN

good job,
i am looking for some good ann algorithms and integration them to my graph framework.
i really want to LiYundi this project for free

3cdbab2#r57157112

https://zilliverse.feishu.cn/docs/doccnE0osweJwlzmtM0DoDwUl7b#PNdSfm
Build Machine is a NUMA

If build time too long, then make build code NUMA-aware.

https://github.com/PwzXxm/BigAnn/blob/master/CMakeLists.txt#L24

• build & release
• assert flag
• add more script
• update README
• Early Input Sanitizer
  • check if index foler is there: clean up it if exists?
  • check e.g.: if d & m == 0

ETA: 09.23 16:00

hi
when i run the build script, billion-scale-benchmark image missed.
Would you please share the right image download address?
Thanks!

https://github.com/shengjun1985/BigAnn/blob/edc3c0817e79d012620fbfe6295c91db2af99e16/src/diskann2.cpp#L1345

Integrate this lib: https://github.com/KaHIP/KaHIP

Check here for the function signature:
https://github.com/PwzXxm/BigAnn/blob/e0524635d7e6c3041b8bd667fdc03a459708047f/include/ivf/clustering.h#L525

centroids: the centroids of the vectors after clustering
assign: the assignment (the corresponding cluster id or bucket id) of vectors in x_in ,
dists: the distance to the centroids

Add commit to this branch:
https://github.com/PwzXxm/BigAnn/tree/graph-partition

1. distance(q, 10-th NN) v.s. centrods distances => distribution
2. x: nprobe, y procentage of hits

zilliz@vm1:~$ uname -a
Linux vm1 5.4.0-1055-azure #57~18.04.1-Ubuntu SMP Fri Jul 16 19:40:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Linux Kernel 5.4.0 liburing is supported.

For example:
https://github.com/shengjun1985/BigAnn/blob/master/include/pq/pq_residual.h#L96

assignee: unknown

• add 10M, 100M
• remove irrelevant py code