Too many open files about ucs HOT 4 CLOSED

For good measure: It's in a branch where I'm experimenting with namespaces, so that may be related.

from ucs.

Not related to namespaces, but fs-cache GC - when run with a cache-size of eg. 1k, it will GC a lot, and eventually it can't keep up with the clients -- half closed connections and open file-handles all over.

go run ./cmd/ucs -quota 100k -verbose
go run ./cmd/ucs-bender #Possibly in separate terminal window
# Now observe the server's epic slowness

(Edited to clarify it's about the cache GC, not Go GC + example)

from ucs.

Thinking it over, it should have some way of ensuring we'll do a GC on/after an upload. Current logic is to check if (quota < size + newly-uploaded-item) { gc() }, which means we'll do a GC on most uploads when we're hovering around the quota.

• Run GC periodically (either timer or every X requests/connections)
• Over-GC (say, 10% or just remove the oldest entry from every bucket)
• Have a lock/queue, so we don't over-schedule GC's (we can upload small items a lot faster than we can GC them)
• More fine-grained locking so GC doesn't block new uploads (goes with locked GC'ing, so we don't have something looking at files while another is deleting them...)

from ucs.

After some thinking / digging it turned out to be caused by a missing Close() call when GC'ing.

Bumping the last comment to a new issue, as it really should be fixed too - see #13

from ucs.