[Bug]: INVERTED scalar filter has low precision in query/search about milvus HOT 7 CLOSED

thanks for reporting the bug, @ghallsimpsons , already fixed in #32858

from milvus.

/assign @longjiquan
please help to take a look, meanwhile, i will try to reproduce it in house

from milvus.

INVERTED

@ghallsimpsons should you use same random number for different fields?
otherwise how did you specify your ground truth?
both index should have 100% recall.

from milvus.

@ghallsimpsons should you use same random number for different fields? otherwise how did you specify your ground truth? both index should have 100% recall.

Hi ~xiaofan-luan, thanks for helping look into this. There is no ground truth here per se, except for what I am requesting via the query. That is, if I perform a search and add the filter inv_index == 1, I would expect every returned row to have inv_index == 1. This is true of the STL index and the no-index case, but not for the inverted index.

from milvus.

could you share you code and what is the result you get?

from milvus.

I have reproduced the issue in house with the code above.

Index no_index: Total of 128 rows
Index no_index: Total of 128 correct rows
Index default_index: Total of 128 rows
Index default_index: Total of 49 correct rows
Index stl_index: Total of 128 rows
Index stl_index: Total of 128 correct rows
Index inv_index: Total of 128 rows
Index inv_index: Total of 49 correct rows

we can see that when filtering with the inverted field, it returns some results that do not in the filter list. e.g.

from milvus.

Very nice, thanks for the quick fix! I'll give it a go again when 2.4.2 is released.

from milvus.