WARNING: nds:loadDatastore marshal gob: type not registered for interface: time.Time about nds HOT 12 CLOSED

Never mind, there must have been some kind of corruption. After flushing memcache, the problem disappeared.

from nds.

Ok, despite continuing to clear my cache, I still keep getting this warning.

from nds.

I'll create a unit test to check. I will possibly need to gob.Register(time.Time{}) on initialisation of the package. This might mean that I need to do the same with ByteString, BlobKey and GeoPoint.

from nds.

How does commit 54e0d82 work? I added gob.Register for the standard types accepted by appengine/datastore entities.

from nds.

Thanks for the amazing response time. The warning disappeared from my logs and it successfully retrieved my data from my cache.

On a cache hit, shouldn't it only perform a single memcache operation, instead of the 4 it performed here?

from nds.

Yes, on a cache hit you should only see a memcache.Get. I don't know why there would be two gets and sets.

On the first get you should see the following as the cache is primed:
memcache Get
memcache Add
memcache Get
datastore Get
memcache Compare And Swap

The current unit tests confirm this behaviour so I'll need to try and create a test to recreate this. Are you using a PropertyLoadSaver? Are you able to tell me what piece of NDS code the memcache.Gets and memcache.Sets is referring to?

from nds.

I'm not using a PropertyLoadSaver here to eliminate the possibility of that causing the error. I'm using a simple Get on a struct with a couple time fields.

from nds.

Here are the four individual stack traces, all back to the single line in my code that calls nds.Get.

Call 1:

Call 2:

Call 3:

Call 4:

from nds.

Thanks and nice catch. This exposes an issue/discrepancy between appengine/memcache and appengine/datastore. If you call appengine/datastore.GetMulti with len(keys) == 0 then it short circuits and doesn't actually make an underlying call to the datastore service. However if I were to call appengine/memcache.Get/Set/AddMulti with len(keys) == 0 it will still call the underlying service.

In your case, NDS is correctly fetching the cached entity in the first memcache.Get. It then continues to execute as expected by calling the other memcache locking code with len(keys) == 0. I know len(keys) == 0 because an underlying datastore.Get call is not recorded on appstats.

This issue does not affect cache consistency - just efficiency. I will create a patch to ensure that nds.GetMulti short circuits when all entities are found in cache.

from nds.

Commit 7bcaf89 should fix this issue. I'll raise the inconsistency between appengine/datastore and appengine/memcache with the App Engine team if you are happy with this fix.

from nds.

Awesome, you have been truly fantastic to work with. I can confirm that this commit fixes the issue. I also tested it including PropertyLoadSaver and that also worked correctly. Great job!

from nds.

Thanks your help @derekperkins with the bug reports. People really expect such a fundamental API like this to be practically bug free - and that is what I am aiming for. Thankfully none of the bugs for a long time have had anything to do with cache consistency!

from nds.