`tree_from_buf` hangs when parsing a small test case about grammar-mutator HOT 4 CLOSED

I've tried the new commit and it worked great.
I had thought this might be related to antlr since afl hanged when calling antlr4::DefaultErrorStrategy::sync, but I wasn't aware of the fact that ambiguous grammar rules could cause trouble.

Thank you so much for your help!

from grammar-mutator.

I will take a look later this week.

from grammar-mutator.

My guess is that your python grammar has inappropriate recursive rules. For example,

"<program>": [["<stmt>", "<NEWLINE>"], ["<stmt>", "<NEWLINE>", "<program>"]],
"<decorators>": [["<decorator>"], ["<decorator>", "<decorators>"]],

The two rules for <program> may result in ambiguity, as they look quite similar. There may exist other recursive rules in your grammar.

I would change them to the following rules to avoid ambiguity:

"<program>": [["<stmt>", "<NEWLINE>", "<program>"], []],
"<decorators>": [["<decorator>", "<decorators>"], []],

This is just my guess, and need to check ANTLR4 documents for more information.

from grammar-mutator.

Hi @OctavioGalland

A temporary workaround is pushed: 4899a2f. You can give it a try.

The new commit fixes a naming issue. Since you are using the grammar generator to generate test cases, the new commit can help you avoid parsing during the initialization stage.

Still, the issue is not fully resolved, as it is actually caused by the ambiguity of the grammar. You can also confirm the infinite loop behavior using ANTLR4:

python grammars/f1_g4_translate.py ./grammars/python.json ./grammars  # Generate Grammar.g4
mkdir temp
cd temp
java -jar ../antlr-4.8-complete.jar ../grammars/Grammar.g4
javac -cp "../antlr-4.8-complete.jar:$CLASSPATH" Grammar*.java

java -cp "../antlr-4.8-complete.jar:$CLASSPATH" org.antlr.v4.gui.TestRig Grammar entry -tree -trace ../in/0
# the program will hang here

To fully address your issue, you may want to reach out to the ANTLR4 community for help to diagnose the root cause of the grammar.

from grammar-mutator.