All memcached tests fail with tarantool 2.10.0-beta1-376-gd2a012455 (master branch) about memcached HOT 3 CLOSED

2021-12-21 16:07:08.342 [1839657] main C> entering the event loop                                                                       
tarantool: /home/sergeyb/sources/MRG/memcached/third_party/small/small/slab_cache.c:58: slab_assert: Assertion `pthread_equal(cache->thread_id, pthread_self())' failed.                                                                                                        
[001] binary/binary-expire.test.py                                                                                                      
[001]                                                                                                                                   
[001] [Instance "binary" killed by signal: 6 (SIGABRT)]                                                                                 
[001] Found assertion fail in the results file [/home/sergeyb/sources/MRG/memcached/test/var/001_binary/binary.log]:                        

Tarantool 2.10.0-beta1-376-gd2a012455

from memcached.

Looks related to #59, we had libsmall update in tarantool recently: tarantool/tarantool#6678.

However after tarantool/tarantool#5932 it should not hit us. At least, not due to symbols clash.

I grepped cord_slab_cache() usage:

$ ag cord_slab_cache
memcached/internal/network.c
34:	mempool_create(&ibuf_pool, cord_slab_cache(), sizeof(struct ibuf));
35:	mempool_create(&obuf_pool, cord_slab_cache(), sizeof(struct obuf));
51:	ibuf_create((struct ibuf *)ibuf, cord_slab_cache(), iobuf_readahead);
60:	obuf_create((struct obuf *)obuf, cord_slab_cache(), iobuf_readahead);

memcached/internal/memcached.c
250:	region_create(&con.gc, cord_slab_cache());

memcached/internal/alloc.c
9:	struct slab *slab = slab_get(cord_slab_cache(), len);
31:	if (ptr) slab_put(cord_slab_cache(), slab_from_data(ptr));

At least we try to use module's region_create() together with tarantool's slab cache. We also try to use memcached's slab_get() / slab_put() with tarantool's slab cache.

from memcached.

The key reason of the fails is curious. Look at 1.1-91-g6bb9a52 in small. It fixes the following lines (the code below is the fixed version):

struct slab_cache {
	<...>
#ifndef NDEBUG // !! it was _NDEBUG
	pthread_t thread_id;
#endif
};

static inline void
slab_cache_set_thread(struct slab_cache *cache)
{
	(void) cache;
#ifndef NDEBUG // !! it was _NDEBUG
	cache->thread_id = pthread_self();
#endif
}

Before the change this code were always compiled (as in Debug as well as in the RelWithDebInfo builds). After tarantool 2.10.0-beta1-315-g173ce9fbb, where this fix lands, the ifndef'ed code present only in the Debug build.

When memcached is built in the Debug configuration the following assertion is in effect:

static inline void
slab_assert(struct slab_cache *cache, struct slab *slab)
{
	<...>
	assert(pthread_equal(cache->thread_id, pthread_self()));
	<...>
}

Since we're using a slab cache from tarantool, which is built as RelWithDebInfo, there is no thread_id field in the structure and it is not filled by actual thread ID. It contains garbage and the assertion fails.

Curious effect of this is that RelWithDebInfo tarantool should work with RelWithDebInfo memcached and Debug tarantool should work with Debug/RelWithDebInfo memcached. IOW, only RelWithDebInfo tarantool plus Debug memcached does not work. At least in context of this issue: I don't know, whether using of tarantool's slab cache in the module leads to other problems.

from memcached.