"unable to open database file" with Python 3.11 and PySide6 threads about python-diskcache HOT 9 OPEN

How many threads are there?

try moving the cache reference to a single global reference. Opening and closing it a gain and again in fast succession from multiple threads is likely to cause issues.

from python-diskcache.

How many threads are there?
Something from 1 to 10, I have one thread per graph so it depend on usage.
The more I have threads the quicker I reach the maximum fd per process.

try moving the cache reference to a single global reference. Opening and closing it a gain and again in fast succession from multiple threads is likely to cause issues.
Unless I missed something it's already what I do.

To give more context, with python 3.10, 3.9 it works perfectly (several days). What I see with Python 3.11 is that sometimes a thread leaks a file descriptor (doesn't look systematic) so I need repeat quite a lot of requests to reach 1024 opened files.
The way it works, is for each graph I have a thread that listen for data requests. A data request correspond to any user interaction with the graph that modifies the X axis range (zoom, pan).
There is only one thread per graph and it is always the same, I mean I create only one thread at graph creation and this thread handles all data requests for this graph and this thread is supposed to use the single shared diskcache instance.

Is that possible that somehow diskcache or sqlite3 reopen the database because it lost the previous fd from within the same thread?

from python-diskcache.

Is that possible that somehow diskcache or sqlite3 reopen the database because it lost the previous fd from within the same thread?

I don’t think so.

Sounds like a problem at a lower level than DiskCache.

from python-diskcache.

@grantjenks thanks for your help, I will continue investigating. I must be able to build a simple reproducer at some point.

from python-diskcache.