[BUG] High RAM usage about whoogle-search HOT 3 CLOSED

Yikes. I tracked it down, looks like the library I was using to make funny rhymes out of the User Agent was extremely inefficient with memory. Bypassing that knocked it down to about 30MB of RAM. All that memory for a stupid joke...tsk tsk. Thanks for pointing that out, will push a fix soon.

from whoogle-search.

$ docker stats --no-stream whooglesearch
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
35b5b8492a1d        whooglesearch       0.03%               25.42MiB / 1.943GiB   1.28%               2.41kB / 15.9kB     0B / 0B             2

Fixed in 445019d, will document in upcoming release in the next few days. Thanks again!

from whoogle-search.

Wow, that's pretty incredible. I wasn't expecting it to be too difficult to fix, but I didn't anticipate it all coming from one dependency. Thanks for the fast turn-around time! I'll keep my eye out for that next release.

I looked into Phyme, and it looks like all that memory is used for the dictionary storage (and resulting lookup structures). Another good reminder that although Python is really convenient to work with, it has some serious costs on the hardware.

from whoogle-search.