tuning the tmpCtx to improve HNSW build performance about pgvector HOT 5 OPEN

@tvondra To be pedantic, what kinds of embeddings are these (dimensions, dimension data type (fp32?)? My rough math says 768-dim, fp32.

Also, this looks really promising 😄 But the reason I'm being pedantic is I think it'd be good to test the contexts across some different dimensionalities. VectorDBBench has a few large datasets for 768/1536 dim, it may be worth to try it out on the 1536 dim to see if you still see a similar gain (my gut says there will be a gain).

from pgvector.

Yes, vector(768) with FP32.

I'd probably expect this to be even more beneficial for larger vectors.

from pgvector.

Hi @tvondra, nice find! From some initial testing (and the profile), it looks like this affects the on-disk phase of the index build and can likely be applied to inserts as well. I also think we may be able to avoid copying vectors entirely if they are outside of the max candidate distance, so will take a look at that as well as part of this.

from pgvector.

Pushed a version of less copying to the hnsw-less-copy branch. Initial results for 100k, 1536-dimension random vectors on my local machine (using 4 processes and 64MB for maintenance_work_mem): 189 sec before, 165 sec after.

Edit: I think there's room for further optimization, as we don't need to load the element at all (for search or builds/inserts) if it's outside the max candidate distance.

from pgvector.

Merged the branch in the commit above. There still may be some room to tune the memory context, but this should get most of the benefit from my testing.

from pgvector.