leveldb.maxopenfiles is not respected about warp10-platform HOT 7 CLOSED

The problem only appears when launching Warp10 with its unit file. When using the warp10 user, the problem does not appear.

from warp10-platform.

How many SST files are there in the leveldb directory? Looks like Warp 10 attempted to redispatch the SST files because it did not have access to the MANIFEST file, this hypothesis would be confirmed if the number of SST files is around 500k.

from warp10-platform.

There are 280224 files.
I ran a leveldb repair a few hours ago. Also, when running Warp 10 with the warp10 user, two CPU cores are maxed out and Warp 10 doesn’t open its HTTP port (I waited for a few hours and it seems stuck), that also happened right after the repair. Maybe it is repair related and I should rename the issue?

from warp10-platform.

Hmmm I think I start to see the problem: Compacting 280212@0 + 0@1 files.

from warp10-platform.

Yes, LevelDB repair is a process which scans all the SST files to determine the start and end keys of each one then puts them all at level 0. The next time LevelDB is open compactions will kick in which will attempt to ventilate the SST files in further levels.

from warp10-platform.

Damn… The compaction seems extremely slow (20k keys every 10 minutes), and I have like 400GB of data. Any tips on how to speed up the process (if it is even possible)?
To get back to the original message, LevelDB tries to open more files than allowed (actually, it seems that it opens the same files several times). Is it only the case during the compaction phase?

from warp10-platform.

The max open files is simply for the normal operations of LevelDB. For compactions it has to open at least N + 1 files when compacting N files.

I've sent you an email asking for you MANIFEST file.

from warp10-platform.