kohler / masstree-beta Goto Github PK
View Code? Open in Web Editor NEWBeta release of Masstree.
License: Other
Beta release of Masstree.
License: Other
This issue is about a key that was inserted successfully to the tree but failed to be retrieved or deleted. Inserting the same key again might succeed.
Scenario:
Thread 1: Inserting a key A. It reaches leaf X (most left leaf) and doesn't find the key A. It's about to insert it to leaf X (before node version validation).
Thread 2: Inserting multiply keys to leaf X and causing a split in a way key A is not belong to this leaf anymore (should be inserted in a leaf to the right).
Thread 2: Deleting all keys in leaf X (Leaf X is now empty).
Thread 1: Notice that the node's version has changed and looking for the next node that should contain the key (in the same layer) but decides to stay in the same leaf while it should have continued to the next leaf.
If I pass nodeparams anything less than 15 (I tried 14 and 13), I get a bunch of errors when running ./mtttest, which look like:
leaf 0x7f573dd57a80: 10 keys, version 68000000, permutation 0b762a4853:91c, parent 0x7f57090cc600, prev 0x7f56e1131700, next 0x7f570606
2780 [ksuf i1]
11495984 = SUBTREE #0/137
leaf 0x7f5735490000: 2 keys, version 0, permutation 01:cba98765432, parent (nil), prev (nil), next (nil)
19 = 1149598420 @2931851292.828826 #0/2
98 = 1149598499 @2931851292.828826 #1/2
1149598577 = 1149598578 @2931851292.828826 #b/9
1149598656 = 1149598657 @2931851292.828826 #7/9
1149598735 = 1149598736 @2931851292.828826 #6/9
11495988 = SUBTREE #2/137
leaf 0x7f56eb82f5c0: 2 keys, version 0, permutation 01:cba98765432, parent (nil), prev (nil), next (nil)
14 = 1149598815 @2931851292.828826 #0/2
93 = 1149598894 @2931851292.828826 #1/2
1149598972 = 1149598973 @2931851292.828826 #a/9
1149599051 = 1149599052 @2931851292.828826 #4/9
1149599888 = 1149599889 @2931851292.828826 #8/9
1149599967 = 1149599968 @2931851292.828826 #5/9
11496000 = SUBTREE #3/137
leaf 0x7f56dbcdb200: 2 keys, version 0, permutation 01:cba98765432, parent (nil), prev (nil), next (nil)
46 = 1149600047 @2931851292.828826 #0/2
7 = 114960008 @2931851292.828826 #1/1
Hi,
I'm trying to run Masstree with the native UDP client
After following the setup guidelines provided in the README, I've used the following to start the server:
./mtd --cores 2 -j 2
and the following to start my client:
./mtclient -s 192.168.1.2 -j 2 -u rw1
The client seems to hang. However, the same test work fine without the -u flag (which I believe would make a TCP client).
I had specified 2 threads and 2 cores, but I'm also curious as to why there are node counts for more than two cores.
Are these bugs, or am I missing something?
Would sincerely appreciate your feedback.
Thanks,
Hamed
Can reproduce with GCC -O3 and Clang -O0.
diff --git a/GNUmakefile.in b/GNUmakefile.in
index 765ae96..746a1f9 100644
--- a/GNUmakefile.in
+++ b/GNUmakefile.in
@@ -64,6 +64,9 @@ jsontest: jsontest.o string.o straccum.o json.o compiler.o
msgpacktest: msgpacktest.o string.o straccum.o json.o compiler.o msgpack.o
$(CXX) $(CXXFLAGS) -o $@ $^ $(MEMMGR) $(LDFLAGS) $(LIBS)
+scantest: scantest.o compiler.o misc.o $(KVTREES) libjson.a
+ $(CXX) $(CXXFLAGS) -o $@ $^ $(MEMMGR) $(LDFLAGS) $(LIBS)
+
config.h: stamp-h
GNUmakefile: GNUmakefile.in config.status
diff --git a/query_masstree.hh b/query_masstree.hh
index 166e104..eb99899 100644
--- a/query_masstree.hh
+++ b/query_masstree.hh
@@ -27,7 +27,6 @@ class query_table {
public:
typedef P parameters_type;
typedef node_base<P> node_type;
- typedef typename node_type::nodeversion_type nodeversion_type;
typedef typename P::threadinfo_type threadinfo;
typedef unlocked_tcursor<P> unlocked_cursor_type;
typedef tcursor<P> cursor_type;
diff --git a/scantest.cc b/scantest.cc
new file mode 100644
index 0000000..8b1ca66
--- /dev/null
+++ b/scantest.cc
@@ -0,0 +1,18 @@
+#include "query_masstree.hh"
+
+using namespace Masstree;
+
+kvepoch_t global_log_epoch = 0;
+volatile uint64_t globalepoch = 1; // global epoch, updated by main thread regularly
+volatile bool recovering = false; // so don't add log entries, and free old value immediately
+kvtimestamp_t initial_timestamp;
+
+int
+main(int argc, char *argv[])
+{
+ (void) argc;
+ (void) argv;
+
+ threadinfo ti;
+ default_table::test(ti);
+}
$ gdb scantest
(gdb) r
Starting program: /vagrant/scantest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
pool_allocate (tag=memtag_masstree_leaf, sz=320, this=0x7fffffffe500) at kvthread.hh:334
334 pool_[nl - 1] = *reinterpret_cast<void **>(p);
(gdb) bt
#0 pool_allocate (tag=memtag_masstree_leaf, sz=320, this=0x7fffffffe500) at kvthread.hh:334
#1 make (ti=..., node_ts=0, ksufsize=0) at masstree_struct.hh:312
#2 make_root (ksufsize=0, ti=..., parent=0x0) at masstree_struct.hh:320
#3 Masstree::basic_table<Masstree::default_query_table_params>::initialize (this=0x7fffffffe1e0, ti=...) at masstree_struct.hh:546
#4 0x000000000040d7eb in initialize (ti=..., this=0x7fffffffe1e0) at query_masstree.hh:45
#5 Masstree::query_table<Masstree::default_query_table_params>::test (ti=...) at query_masstree.cc:294
#6 0x0000000000401cff in main (argc=<optimized out>, argv=<optimized out>) at scantest.cc:17
(gdb)
For what it's worth, one needs to add -lexecinfo
to $LIB
to build on OpenBSD and FreeBSD (and, I would assume, NetBSD). This is because backtrace(3)
and friends are stored there rather than in libc. I tried to find an elegant way to add this to configure.ac
, but I have little experience with autoconf and it seems that masstree doesn't yet use the features necessary for AC_CANONICAL_TARGET
or AC_CANONICAL_HOST
.
If there's an easy way of adding it, it'd be appreciated. Otherwise, it's easy to manually patch into GNUMakefile.in
.
The following two lines of code in scan seems to be giving us issues:
masstree-beta/masstree_scan.hh
Lines 270 to 271 in 6b6c118
Basically if the content of n_->keylenx_
changes between the check and the n_->ksuf()
call, the precondition check in ksuf()
might fail because the precondition check loads the latest value. Is this a bug?
This issue is about using reverse scanner on a tree with an empty trie node (b+ tree).
Scenario:
Example Tree structure(After delete all keys):
green border: leaf node
coral color: layer root
I wonder if we can do some checks and issue a gc_layer during remove_leaf.
I try to fix it like this: https://github.com/jimdb-org/masstree-beta/commit/eabe31105a283a956ca84e309b9221add8f71575 , and the problem goes away.
This issue might occur when a layer is being removed.
Scenario:
Thread 1: Deleting last key "1111111122222222" in the 2nd layer, and add request to remove the layer which continue from "11111111"
Thread 1: Cleanup is called, and layer is being removed. in function gc_layer, the leaf's node was just taken
Thread 2: Inserting key "1111111122222222", and currently waiting on the same leaf's lock (in function find_locked)
Thread 3: Inserting key "1111111133333333", and currently waiting on the same leaf's lock (in function find_locked)
Thread 1: Mark leaf as "deleted layer" and unlock it
Thread 2: Acquire the leaf's lock, check leaf's state and find out the layer was removed. It continues without unlocking the leaf.
Thread 3: Stuck in dead-lock
file configure is the same as configure.ac???
https://github.com/kohler/masstree-beta/blob/master/string.cc#L285
I'm not sure there's a good cross-platform macro check, but a cheap workaround for me is to include && !arm
README makes mention of it at the end
I may found bugs. Could you read this comment?
Line 446 in 4f4bc58
Then, comparing method is forced this line.
Line 151 in cef4cc4
I think there is a problem.
If we use the key whose length is 9, or 17, or..., which isn't divisible by 8, it causes segv.
Is this right?
Sincerely yours,
Takayuki
masstree-beta/masstree_remove.hh
Line 68 in cfc62fa
It is unsafe to make_layer_root without locking the node child.
In the implementation of make_layer_root, setting the root_bit is not atomic.
(Even if it is atomic, there will also be other problem)
Line 274 in cfc62fa
Deadlock Scenario:
Thread A:Inserting a key into the node child, lock child (lock_bit is set now)
Thread B: doing gc_layer, runnng inside mark_root
, load v_ and calculate tmpv = v_ | P::root_bit ( lock_bit is set in tmpv)
Thread A: insert finished, unlock child (lock_bit is unset now)
Thread B: assign tmpv back to v_ , then the node child's lock_bit is set forever
Since the locking order in the same layer is from bottom to up, can a try_lock operation fix this?
My tentative solution: https://github.com/chubaodb/masstree-beta/commit/64bad5ebe3a2dbf998ff2e0ac9c24548cf0d226f
This issue is about a key that was inserted successfully to the tree but failed to be retrieved or deleted. Inserting the same key again might succeed.
Scenario:
Thread 1: Inserting key A. Traversing down the tree looking for the right leaf
Thread 2: Deleting key B, where B > A and keys share partial\full internode route to the leafs.
Thread 2: Deleting key B, which is the last key in leaf X (Leaf X is now empty).
Thread 2: Deleting leaf X and redirect new key boundary.
Thread 1: Finish insert key A successfully but the key is not reachable.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.