Performance of the search index seems to be rather slow (multiple seconds until the re

Looked at it and found the following: Base model (Person) and

Performance of search index (haystack/woosh),about basxsoftwareassociation/basxconnect

Comments (1)

saemideluxe commented on May 16, 2024

Looked at it and found the following:

Base model (Person) and specific model (NaturalPerson, LegalPerson, Personassociation) have been indexed, even thoug the base model is enough
View was not generating lazy results (as a generator) even though it is easy possible and might improve performance a bit
After fixing the two issues above I profiled with cprofile
Most time is spent in disc-io functions of whoosh and in sqlite function calls
Whoosh (as a python-only search engine) seems to have slightly worse performance than Xapian (C++): https://github.com/whoosh-community/whoosh/wiki/Benchmarks. However the difference in performance is not that severe and it is not worth thinking about replacing it due to binary dependencies and setup issues (we tried xapian already with haystack).

I think this is enough for now. There are options to push the performance, e.g. creating the search-index in-memory or limiting number of search results. However I think the performance is now acceptable and with the newly added loading indicator even longer loading times are acceptable for end users.

from basxconnect.

Recommend Projects