Create a persistent cache for ffprobe data about dms HOT 6 CLOSED

From Igor Shuvalov on 2013-09-21 17:52:54+00:00

you don't like RoninDev/dms-cache ?

from dms.

From Matt Joiner on 2013-09-22 08:24:52+00:00

I'll reevaluate it.

from dms.

From Matt Joiner on 2013-10-03 07:19:14+00:00

@ronindev , my concern is that as soon as the cache becomes persistent, we have the problem that we're reinventing the wheel. That is to say that the cache as it is in your fork is unbounded, and not generic.

I wonder if we should make use of some other technology, such as redis or similar, and make use of their ability to limit the cache size etc.

Another thought is store the metadata in a hidden file next to the file it describes thereby having the metadata deleted when the directory is removed, however that requires write access to the shared path, which can't be assumed.

Let me know what you think, I'm open to ideas here.

from dms.

From Matt Joiner on 2013-10-03 08:28:49+00:00

I've done some poking around. I was going to go with groupcache with NoPeers but it doesn't provide access to the underlying cache, which is LRU anyway. I think the vitess lru cache is the way to go, as the items are accessible and so can be marshalled to and from disk easily.

from dms.

From Matt Joiner on 2013-10-04 19:39:35+00:00

I'm implementing a persistent random replacement cache in the branch persistent-ffprobe-cache. Let me know what you think.

from dms.

From Matt Joiner on 2013-12-09 08:52:24+00:00

Implemented a random-replacement persistent cache.

from dms.