Open files never closed about kfs HOT 2 CLOSED

Note: As soon as I get the first contract offer I will open all leveldbs one by one. A few minutes later I have 1028 open files and they are never closed as well. I would expect that this is needed to get the used space once. Every new request should not open all leveldbs again. It would be better to cache the size of each leveldb and update on change.

root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
350
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
398
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
427
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
450
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
469
root@storj:~# grep 'download\|upload\|contract offer' .storjshare/storjshare/logs/188071ba7cfd974a9e47b59e24b0737ebf845db3.log
{"level":"debug","message":"received contract offer...","timestamp":"2017-02-28T16:16:01.155Z"}
{"level":"debug","message":"received contract offer...","timestamp":"2017-02-28T16:16:14.497Z"}
{"level":"debug","message":"received contract offer...","timestamp":"2017-02-28T16:16:24.483Z"}
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
637
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
689
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
736
root@storj:~# ls -l /proc/416/fd | grep 'storjshare' | wc -l
1028

I am not able to proof that expectation. Feel free to fix it as well. I will test it as soon as unused leveldbs are getting closed. That will allow me to see the result after the second contract offer.

from kfs.

After some testing and one small addition to KFS in v3.1.3, I can confirm that this issue is in storj-lib and not in KFS.

Storj is using Btable#stat() inefficiently. It never calls it with the file key (which will only open the single Sbucket the corresponds to that key), so instead it measures the entire store, opening all of the buckets as you observed.

I can also confirm that KFS does emit the idle event and does close the bucket afterwards. However, the way the Storj is keeping track of capacity available, it is measuring the entire store after every contract save/update/etc - this causes all the buckets to stay open, since every time a PUBLISH message is received, we call that causing the pending operations for each bucket to be > 0 when the idle state is checked thus leaving the bucket open.

Fixing this will require some refactoring to the StorageManager and StorageAdapter classes in Storj. Let's reopen the issue in core and we will work on it this week.

from kfs.