multi-threaded dictionary crash about link-grammar HOT 18 CLOSED

Above stack trace fixed in #1380 Now I get

Thread 103 "cogserv:eval" received signal SIGSEGV, Segmentation fau
#0  pool_alloc_vec (mp=0x7fff4c0155d0, vecsize=vecsize@entry=1)
    at ../../link-grammar/memory-pool.c:189
#1  0x00007fffecbaad75 in pool_alloc (mp=<optimized out>)
    at ../../link-grammar/memory-pool.h:116
#2  Exp_create (mp=<optimized out>)
    at ../../link-grammar/dict-ram/dict-ram.c:442
#3  make_connector_node (dict=dict@entry=0x7fff4c00ca60, mp=<optimized out>, 
    linktype=0x7fff22ffbc70 "DPL", dir=<optimized out>, 
    multi=multi@entry=false) at ../../link-grammar/dict-ram/dict-ram.c:527

from link-grammar.

Above fixed in #1381 ... Next up,the string-set:

#0  find_place (ss=0x7fff50001040, h=940135772, str=0x7fff6f7f4c70 "GLKDB")
    at ../../link-grammar/string-set.c:149
#1  string_set_add (source_string=source_string@entry=0x7fff6f7f4c70 "GLKDB", 
    ss=0x7fff50001040) at ../../link-grammar/string-set.c:189
#2  0x00007fffecbaaefc in make_connector_node (
    dict=dict@entry=0x7fff5000ce90, mp=0x7fff50015be0, 
    linktype=0x7fff6f7f4c70 "GLKDB", dir=<optimized out>, 
    multi=multi@entry=false) at ../../link-grammar/dict-ram/dict-ram.c:530

from link-grammar.

Above fixed in #1383 Next one is this:

#0  sort_condesc_by_uc_constring (dict=dict@entry=0x7fff5000cea0)
    at ../../link-grammar/connectors.c:429
#1  0x00007fffecb90f96 in condesc_setup (dict=0x7fff5000cea0)
    at ../../link-grammar/connectors.c:590
#2  0x00007fffecb8f715 in sentence_parse (sent=sent@entry=0x7fff10002dd0, 
    opts=opts@entry=0x7fff10006890) at ../../link-grammar/api.c:692

from link-grammar.

Above is fixed by #1384

from link-grammar.

I'm starting to think that many of the above fine-grained locks could be removed and replaced by a more coarse-grained lock in dict-atomese. This is because there are still more races, which need the coarse-grained lock to fix. Which makes the fin-grained locks pointless...

from link-grammar.

The one-big-lock in #1387 allows:

• #1383 reverted in #1386

• ---

• #1381 reverted in #1388

• ---

• #1380 reverted in #1389

• ---

• #1384 reverted in #1390

from link-grammar.

Nope. Still have issues.

#0  sort_condesc_by_uc_constring (dict=0x55598d237940)
    at ../../link-grammar/connectors.c:429
#1  condesc_setup (dict=0x55598d237940) at ../../link-grammar/connectors.c:581
#2  0x00007fffe836c735 in sentence_parse (sent=sent@entry=0x7fff100027a0, 
    opts=opts@entry=0x7fff10001450) at ../../link-grammar/api.c:692

from link-grammar.

Do you have a full backtrace?

from link-grammar.

Do you have a full backtrace?

I'm working on a redesign to take care of this..

from link-grammar.

The redesign will notify the dictionary when parsing starts and stops. This will allow the dictionary to know what the words are in the current sentence, and do special per-sentence stuff.

from link-grammar.

The redesign that (I hope) fixes the last stack trace was #1394 ... all the earlier pull reqs were cleanup to enable this change. I'm hoping this is the last one...

from link-grammar.

As of commit f52ac71, there is a strange parsing problem, that doesn't seem related to the deduplicate_linkages() bugs I pointed out in #1378: Many sentences silently get ignored (no parsing and no error).

E.g:
He said that, finding that it was impossible to get work as a waiter, he would work as a janitor

I don't have any idea of a possible reason (haven't bisected yet), but it seems it depends on the number of words - if you remove one by one from the end you finally get a response, but then if you add any random word then you get nothing. I'm still investigating the apparent problems in removing duplicate parses, so for now I'm just reporting this error without looking deeply into it.

BTW, there is also seems to be a performance hit (+~10% on short sentences, +~7% on fix-long, +6%% on pandp-union). I wonder why, as you have removed the fine-grain locks (I will try to profile it latter).

from link-grammar.

Thank you. It's no longer crashing, but it is behaving very oddly. For me, the vast majority of sentences seem to have gone AWOL. But I'm not using the command line tool, so its harder for me to debug.

from link-grammar.

One obvious culprit is #1385 -- it actually changes things.

from link-grammar.

The other is to revert #1378 until such time that the several issues you mention are fixed. ...

from link-grammar.

what a mess. "move fast and break things".

from link-grammar.

The dedupe is the cause of the missing linkages, and I've got a fix for that.

from link-grammar.

OK, once again -- I think everything is now fixed, and all commentary has been handled. You mentioned slowdowns ... I too measure things and keep poor notes.

The only slowdown I saw was from #1103 but my notes are not clear it that was recovered or not. I've been observing an 8% slowdown consistently, ever since #1103 But this could also be explained by other changes, too.

from link-grammar.