Segfault in cache::get/has about carmen-cache HOT 4 CLOSED

If you had just mentioned that there was a race condition somewhere in carmen-cache leading to invalid reads, I would have guessed that it was in coalesce, which calls __get (or __getTruncated) in a thread other than the main thread, while the main thread might potentially be mutating the cache structures __get would access, either with calls to unload, or with calls to loadSync which cause cache eviction (as the cache has a limited size). In particular, I think there's a race condition opportunity here between the cache membership check and the cache fetch in _get, where if _get is called from coalesce it might be running in the thread pool, and in the intervening time the entry could get evicted or unloaded from the cache in the main thread.

But that's not what's going on here, if the crash is in the has and _get NAN_METHODs. has, _get, loadSync, and unload should all only get called from JS, so they should all be running on the main thread, and the latter two are the only things that might invalidate cache contents as far as I know, so I'm pretty confused as to how mutation could be occurring during a call to has or _get, as I don't think it should be possible for any of the mutating functions to be called at the same time as those two. It's super weird.

from carmen-cache.

Thank you for the thoughts! Right in this case neither of the asynchronous functions are being invoked (coalesce or merge). But i'll be digging into this more late tonight or tomorrow am to see if i can surface any more clues.

from carmen-cache.

Recapping: because this is so mysterious I suspect memory corruption is causing it (so the bug might be elsewhere and only manifesting in get/has calls because they are called so often and will fall prey to memory being messed up elsewhere). Therefore, my game plan is:

• Solve all the overflow issues flags by the sanitizers first: #75, #79 and #83
• Instrument libc++ with the address sanitizer
• Then revisit this test to try to replicate again.

from carmen-cache.

Recapping here:

Solve all the overflow issues flags by the sanitizers first: #75, #79 and #83

This was done in #86

Instrument libc++ with the address sanitizer

Still not go around to this. Ticketed at mapbox/mason#411

Then revisit this test to try to replicate again.

No plans to at this point. Going to hope that the actual bug was fixed in #86.

from carmen-cache.