Dangling reference on `cache_evict` when using referenced keys about nebulex HOT 4 OPEN

Hey @ahovgaard!!

Thanks for spotting this issue and for the very detailed information. I know the references are not being removed on the evict decorator (just the given key but not its references), but you're right, it is an issue when a referencing key is updated, like you said, we end up with dangling keys in the cache, which I agree is not desirable. Let me look into it, and will try to come back with a solution soon.

from nebulex.

Deleting references during GC or backtracking them can be a solution

from nebulex.

I've been thinking about this and the best way to address it, but there is no quick fix, and any solution has pros and cons. So, let me break down the problem and address it incrementally.

• The dangling keys. IMHO, I wouldn't consider this critical because there are some ways to handle it (will explain later). I agree that is not ideal, but providing a solution like backtracking the key references may bring also downsides as well in terms of memory and performance. So, I'd like to explore other alternatives I have in mind, but that will be for Nebulex v3; which is the highest priority now. In the meantime, to address this, there are a few things we can do. The first one (and perhaps the simplest one), is using the :ttl option. Another alternative, and the one I recommend and use myself, is having a separate cache for keeping the references only, a local cache with the local adapter for instance. This way, one could provide a different eviction or GC configuration, to run the GC more often and keep the references cache clean.

• Inconsistencies due to a dangling key (I'm talking about this: In step 7, a new record is written to the DB (say from another machine running the app) using the old ID 123. When we read the record using that ID in step 9, the previously dangling reference from "[email protected]" now points to the new record with the mismatched email.). This issue, on the contrary, I think is critical to fix, because we don't want to retrieve an inconsistent value. I've been working on something to address it. The idea is to leverage the match function when retrieving a reference to ensure the value is what we're expecting, and also be able to set a TTL for a reference (plus other stuff). However, it will be available in Nebulex v3; like I mentioned before, Nebulex v3 is the top priority now. I will keep you posted.

• Lastly, and perhaps the main problem, is the inaccurate and misleading documentation. I will fix the documentation based on what we have discussed here, explaining the issue and the possible immediate alternatives to address it.

Thank you very much for all the comments, suggestions, and so on, they're very well appreciated. I'll keep you posted about the progress.

from nebulex.

I've pushed a change fixing the docs about the cache_evict decorator.

Also, as mentioned before, the inconsistency issue will be addressed on Nebulex v3 (https://github.com/cabol/nebulex/blob/v3.0.0-dev/lib/nebulex/caching/decorators.ex#L742-L805).

from nebulex.