getent: the module is slow about syslog-ng-incubator HOT 10 CLOSED

Well, it does memory allocation, lookup and formatting every single time. I suppose using nsdc or similar for caching may make it faster, and with some tricks, we could make it possible to only allocate once (but then we'd have a lock, so that likely wouldn't buy us much).

I'll try to do some profiling to see which part is the slowest. By the way, are you using LDAP or something similarly expensive? And which getent function is slow? passwd & group?

from syslog-ng-incubator.

I do use nscd and I was using getent protocols.
I'll test if others are faster tomorrow.
geoip is much faster

from syslog-ng-incubator.

GeoIP loads the database once on startup (or reload), and only uses memory after that, if I remember correctly. I suppose we could do the same with getent, but that's risky (if you have an LDAP with 30k+ users, startup would be incredibly slow...).

from syslog-ng-incubator.

Hm... or, we could cache results ourselves. I'll try to do some profiling and see if caching helps.

from syslog-ng-incubator.

Awesome, thanks!

from syslog-ng-incubator.

I think this question is answered, so I close this Issue

from syslog-ng-incubator.

@algernon proposed to do some tests, but this has not been addressed yet

from syslog-ng-incubator.

Sorry, I issue reopened

from syslog-ng-incubator.

Not sure when (or in all honesty, if) I will have time to do any tests.

from syslog-ng-incubator.

genet module is removed from the incubator and merged into syslog-ng master.

If you still have concerns with the speed of the getent module, please open a new ticket in the syslog-ng project.

from syslog-ng-incubator.