Socket leakage about snimpy HOT 8 OPEN

Which Snimpy version are you using? Are you using SNMPv2 or SNMPv3? Is the threadpool reusing threads or just spawning new ones?

from snimpy.

I updated the version of snimpy from 0.8.2 to 0.8.8 serveral hours ago.

SNMPv2c.

The threadpool reusing threads.

from snimpy.

I have to update the information I provided:

The number of open udp ports does not increase again. I'm not sure whether the upgrade of snimpy causes it.

What's more: I have 100 worker thread and now I have 100 open port. Maybe there is some relation between the two number?

from snimpy.

Before 0.8.6, Snimpy was reinitializing the whole SNMP stack for each manager. I am not surprised that it could have been the source of your problem. Now, Snimpy does this initialization only once per thread. So, it is expected that you have one open port for each thread (is that a problem for you?).

from snimpy.

It's a problem but not so serious.

Is it possible to release unused port? After all, fd is limited.

from snimpy.

As I can see, manually releasing resource is also acceptable.

from snimpy.

It's possible but it's conflicting with another bug. Snimpy relies on PySNMP. It uses some high level interface ("command generator"). This high level interface is not thread safe, so we need one instance of it in each thread. However, initializing this interface is memory and CPU-hungry. Moreover, it seems that it's not possible to reclaim all the resources that were allocated (hence the leak you observed, plus the memory leak in #33). So, I don't see a way around that.

The default FD limit is 1024, so with 100, you seem to be safe.

Otherwise, maybe you can use a dedicated (smaller) threadpool for SNMP?

In the future, I may switch back to NetSNMP because of bugs like this (notably, for SNMPv3 where I need a command generator for each manager) but also for performance.

from snimpy.

Well, thanks for your graceful contribution. I will continue to keep sight
over snimpy.

On Sun, Jan 17, 2016, 8:20 PM Vincent Bernat [email protected]
wrote:

It's possible but it's conflicting with another bug. Snimpy relies on
PySNMP. It uses some high level interface ("command generator"). This high
level interface is not thread safe, so we need one instance of it in each
thread. However, initializing this interface is memory and CPU-hungry.
Moreover, it seems that it's not possible to reclaim all the resources that
were allocated (hence the leak you observed, plus the memory leak in #33
#33). So, I don't see a
way to do that.

The default FD limit is 1024, so with 100, you seem to be safe.

Otherwise, maybe you can use a dedicated (smaller) threadpool for SNMP?

In the future, I may switch back to NetSNMP because of bugs like this
(notably, for SNMPv3 where I need a command generator for each manager) but
also for performance.

—
Reply to this email directly or view it on GitHub
#48 (comment)
.

from snimpy.