[go target] Build errors with PartiQL parser about antlr4 HOT 8 OPEN

I am reworking the Go visitor code as there are some other issues. Though it generally works, there are some issues that need manual intervention, which is not ideal. It may take a week or so to get this ready for release. though.

Looks like you are using the Listener here though, which does work, so I woudl need to see your code to work out what is wrong here.

Please don't use images for text errors as they are hard to read.

from antlr4.

I have several comments on this grammar. There are several problems.

• grammar PartiQL; is a declaration for a combined grammar. The grammar works, but I've seen similar situations where it doesn't. I would recommend changing the declaration to parser grammar PartiQLParser;.
• options { caseInsentive=true; } has no meaning in a combined or parser grammar. Remove.
• statement is a start rule (i.e., a rule that terminates with EOF), but it is also used in root. This could be a problem if the statement is followed by any terminal or non-terminal symbol in some rule, e.g., root. (What token can occur after the end of file?) The Antlr tool will not warn if you try.
• There are two unused parser rules, tableName and tableconstrantName. The rules should be deleted.

$ bash /c/Users/Kenne/Documents/GitHub/g4-scripts/find-unused-parser-symbols.sh  *.g4
Arguments were provided.
Finding unused parser symbols in grammars...
PartiQLParser.g4:L77: tableName : symbolPrimitive;
                      ^
PartiQLParser.g4:L78: tableConstraintName : symbolPrimitive;
                      ^
05/27-12:38:16 ~/issues/g4-4632
$ grep tableName *.g4
PartiQLParser.g4:tableName : symbolPrimitive;
05/27-12:38:30 ~/issues/g4-4632
$ grep tablecontraintName *.g4
05/27-12:38:43 ~/issues/g4-4632

• There are many lexer symbols declared that are not used at all in the parser grammar, not explicitly (via name, e.g., ABSOLUTE is not on the right-hand side of a parser rule), implicitly (via a string literal, e.g.,'absolute' in the right-hand side of a parser rule), or with a dot in the right-hand side of a parser rule.

$ bash /c/Users/Kenne/Documents/GitHub/g4-scripts/find-unused-lexer-symbols.sh  *.g4
Arguments were provided.
Finding unused lexer symbols in grammars...
1st pass: fold string literals in parser rules to make explicit lexer rule used.
2nd pass: find unreferenced lexer rule symbols.
PartiQLLexer.g4:ABSOLUTE
PartiQLLexer.g4:ACTION
PartiQLLexer.g4:ADD
PartiQLLexer.g4:ALLOCATE
PartiQLLexer.g4:ALTER
PartiQLLexer.g4:ARE
PartiQLLexer.g4:ASSERTION
PartiQLLexer.g4:AUTHORIZATION
PartiQLLexer.g4:BEGIN
PartiQLLexer.g4:BIT
PartiQLLexer.g4:CASCADE
PartiQLLexer.g4:CASCADED
PartiQLLexer.g4:CATALOG
PartiQLLexer.g4:CHECK
PartiQLLexer.g4:CLOSE
PartiQLLexer.g4:COLLATE
PartiQLLexer.g4:COLLATION
PartiQLLexer.g4:COLUMN
PartiQLLexer.g4:COMMIT
PartiQLLexer.g4:CONNECT
PartiQLLexer.g4:CONNECTION
PartiQLLexer.g4:CONSTRAINTS
PartiQLLexer.g4:CONTINUE
PartiQLLexer.g4:CONVERT
PartiQLLexer.g4:CORRESPONDING
PartiQLLexer.g4:CURRENT
PartiQLLexer.g4:CURRENT_TIME
PartiQLLexer.g4:CURRENT_TIMESTAMP
PartiQLLexer.g4:CURSOR
PartiQLLexer.g4:DEALLOCATE
PartiQLLexer.g4:DECLARE
PartiQLLexer.g4:DEFAULT
PartiQLLexer.g4:DEFERRABLE
PartiQLLexer.g4:DEFERRED
PartiQLLexer.g4:DESCRIBE
PartiQLLexer.g4:DESCRIPTOR
PartiQLLexer.g4:DIAGNOSTICS
PartiQLLexer.g4:DISCONNECT
PartiQLLexer.g4:DOMAIN
PartiQLLexer.g4:END_EXEC
PartiQLLexer.g4:EXCEPTION
PartiQLLexer.g4:EXECUTE
PartiQLLexer.g4:EXTERNAL
PartiQLLexer.g4:FETCH
PartiQLLexer.g4:FOREIGN
PartiQLLexer.g4:FOUND
PartiQLLexer.g4:GET
PartiQLLexer.g4:GLOBAL
PartiQLLexer.g4:GO
PartiQLLexer.g4:GOTO
PartiQLLexer.g4:GRANT
PartiQLLexer.g4:IDENTITY
PartiQLLexer.g4:IMMEDIATE
PartiQLLexer.g4:INDICATOR
PartiQLLexer.g4:INITIALLY
PartiQLLexer.g4:INPUT
PartiQLLexer.g4:INSENSITIVE
PartiQLLexer.g4:INTERVAL
PartiQLLexer.g4:ISOLATION
PartiQLLexer.g4:KEY
PartiQLLexer.g4:LANGUAGE
PartiQLLexer.g4:LATERAL
PartiQLLexer.g4:LEVEL
PartiQLLexer.g4:LOCAL
PartiQLLexer.g4:MODULE
PartiQLLexer.g4:NAMES
PartiQLLexer.g4:NATIONAL
PartiQLLexer.g4:NATURAL
PartiQLLexer.g4:NCHAR
PartiQLLexer.g4:NEXT
PartiQLLexer.g4:NO
PartiQLLexer.g4:OF
PartiQLLexer.g4:ONLY
PartiQLLexer.g4:OPEN
PartiQLLexer.g4:OPTION
PartiQLLexer.g4:OUTPUT
PartiQLLexer.g4:OVERLAPS
PartiQLLexer.g4:PAD
PartiQLLexer.g4:PREPARE
PartiQLLexer.g4:PRESERVE
PartiQLLexer.g4:PRIMARY
PartiQLLexer.g4:PRIOR
PartiQLLexer.g4:PRIVILEGES
PartiQLLexer.g4:PROCEDURE
PartiQLLexer.g4:PUBLIC
PartiQLLexer.g4:READ
PartiQLLexer.g4:REFERENCES
PartiQLLexer.g4:RELATIVE
PartiQLLexer.g4:RESTRICT
PartiQLLexer.g4:REVOKE
PartiQLLexer.g4:ROLLBACK
PartiQLLexer.g4:ROWS
PartiQLLexer.g4:SCHEMA
PartiQLLexer.g4:SCROLL
PartiQLLexer.g4:SECTION
PartiQLLexer.g4:SESSION
PartiQLLexer.g4:SESSION_USER
PartiQLLexer.g4:SPACE
PartiQLLexer.g4:SQL
PartiQLLexer.g4:SQLCODE
PartiQLLexer.g4:SQLERROR
PartiQLLexer.g4:SQLSTATE
PartiQLLexer.g4:SYSTEM_USER
PartiQLLexer.g4:TEMPORARY
PartiQLLexer.g4:TO
PartiQLLexer.g4:TRANSACTION
PartiQLLexer.g4:TRANSLATE
PartiQLLexer.g4:TRANSLATION
PartiQLLexer.g4:UNIQUE
PartiQLLexer.g4:UNKNOWN
PartiQLLexer.g4:USAGE
PartiQLLexer.g4:USER
PartiQLLexer.g4:USING
PartiQLLexer.g4:VIEW
PartiQLLexer.g4:WHENEVER
PartiQLLexer.g4:WORK
PartiQLLexer.g4:WRITE
PartiQLLexer.g4:CARET
PartiQLLexer.g4:UNRECOGNIZED
PartiQLLexer.g4:LETTER
PartiQLLexer.g4:LETTER_NOT
05/27-12:40:26 ~/issues/g4-4632
$

• The grammar contains a "symbol conflict". This is why it does not compile for Go. A symbol conflict is an identifier in a rule that causes a compilation error. The "parent" attribute is causing the problem. The fix to rename the attribute: parent= should be changed to parent_= (eight occurrences).

Correcting all these errors, we see the Go target work with this grammar.

PartiQLLexer.g4.txt

PartiQLParser.g4.txt

Input:
ex.txt

# A generated parser was built using trgen. Here the executable is called on input.
$ ./Test.exe ../examples/ex.txt
Go 0 ../examples/ex.txt success 0.025
Total Time: 0.027 s
05/27-13:33:33 ~/issues/g4-4632/Generated-Go

from antlr4.

First, I am curious about this: parent is neither a rule name in the parser nor a token name in the tokens. Is it reserved by ANTLR4?

To see what is happening, let's rename all but the first of the attribute "parent=" to "parent_=" (in parser rule exprOr), then do a build.

$ make
bash build.sh
# example.com/myparser/parser
parser\partiql_parser.go:22846:9: cannot use s (variable of type *ExprOrBaseContext) as antlr.RuleContext value in return statement: *ExprOrBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
parser\partiql_parser.go:22893:13: cannot use NewExprOrBaseContext(p, localctx) (value of type *ExprOrBaseContext) as IExprOrContext value in assignment: *ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
parser\partiql_parser.go:22902:3: impossible type assertion: localctx.(*ExprOrBaseContext)
        *ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
mingw32-make: *** [makefile:3: build] Error 1
05/30-06:16:59 ~/issues/a4-4632/go/Generated-Go

For attribute parent, there has to be a function declared to access the attribute. Antlr Tool does this.

func (s *ExprOrBaseContext) GetParent() IExprAndContext { return s.parent }

func (s *ExprOrBaseContext) SetParent(v IExprAndContext) { s.parent = v }

For the renamed attribute parent_, the code is similar, but critically with a different name.

func (s *ExprAndBaseContext) GetParent_() IExprNotContext { return s.parent_ }

func (s *ExprAndBaseContext) SetParent_(v IExprNotContext) { s.parent_ = v }

However, the Antlr Go Runtime also defines functions GetParent() and SetParent().

Second, I found a related issue mentioned symbol conflict. However, why don't ANTLR tools raise any warning or error in this case?

The Antlr Tool cannot apriori warn there is a conflict because it doesn't do a build. It only "knows" when the symbol is in a list of symbol names that are known to cause build problems. That table is here:

That table does not contain "parent". It does contain "GetParent" however.

The Antlr tool does not warn because it just internally renames symbol conflicts before code generation. Other targets may still work with the original attribute name. The requirement for this solution was to not break existing parsers that use Antlr. We could have implemented a different solution to the problem by adding an option to generate code with a unique string appended to symbol names for other targets, which would have not broken old, working code for the original target. But that was not done here.

I have a script (https://github.com/kaby76/g4-scripts/blob/e5ca411ddbf8eb9fb97e986c99d124991e839899/find-known-conflicts.sh) that parses the Antlr4 grammar and checks all symbols against all known conflicts, but it's hard to keep it up to date. We can't predict when there is a conflict.

from antlr4.

I am reworking the Go visitor code as there are some other issues. Though it generally works, there are some issues that need manual intervention, which is not ideal. It may take a week or so to get this ready for release. though.

Looks like you are using the Listener here though, which does work, so I woudl need to see your code to work out what is wrong here.

Please don't use images for text errors as they are hard to read.

Thank for your reply! I remove the image from issue description and put text instead.

from antlr4.

options { caseInsentive=true; } has no meaning in a combined or parser grammar. Remove.

Incorrect. The option works in a combined grammar (for lexer rules). See testCaseInsensitiveInCombinedGrammar test for example.

Also, it could be actual in parser grammar if Token value comparators feature is implemented (not now).

from antlr4.

Incorrect. The option works in a combined grammar (for lexer rules).

Sorry, what I meant is for this grammar.

Even though PartiQL.g4 has a grammarType without "parser", it is essentially a parser grammar because options { tokensVocab=...; } is set, and there are no lexer rules. However, technically, it's still a combined grammar because one could add lexer rules to PartiQL.g4, run it through the Antlr4 Tool to not only generate a parser, but a second lexer class: PartiQLLexer.java. But then, it's unclear how to do the lexing because PartiQLTokens does not reference PartiQLLexer, PartiQLLexer does not reference PartQLTokens,new CommonTokenStream(lexer) can be passed only one lexer, and new PartQLParser(TokenStream input) takes only one token stream. It is also the case that the vocab and tables describing the rules in PartiQLTokens are not merged into PartiQLLexer, nor vice versa as PartiQLTokens was generated first. It does not work if the parse references both vocabs.

Rather than be exposed to this rabbit hole, a parser grammar should be declared as parser grammar PartiQLParser;.

from antlr4.

The grammar contains a "symbol conflict". This is why it does not compile for Go. A symbol conflict is an identifier in a rule that causes a compilation error. The "parent" attribute is causing the problem.

First, I am curious about this: parent is neither a rule name in the parser nor a token name in the tokens. Is it reserved by ANTLR4?

Second, I found a related issue mentioned symbol conflict. However, why don't ANTLR tools raise any warning or error in this case?

BTW, I can get the correct go target after renamming parent= to parent_=. Thanks a lot!

from antlr4.

I get it. If we name it parent, the generated access code is GetParent(), which conflicts with the function in the embedded struct BaseParserRuleContext. Thank you for your attention to detail and patience.

from antlr4.