Code Monkey home page Code Monkey logo

Comments (14)

Thomasb81 avatar Thomasb81 commented on August 15, 2024

My mistake, apparently it is only an informative message. But running tests coming with runtime show them.

from antlr4.

kaby76 avatar kaby76 commented on August 15, 2024

Actually, it's worse than this one issue. Not only are the generated files inconsistent in timestamps/versions, the XPathLexer.g4's are slightly different, and not just in target-specific code, or symbol renaming because target-agnostic format is not followed. I see a token range difference in the .g4 grammar.

$ find . -name XPathLexer.\*
./Cpp/runtime/src/tree/xpath/XPathLexer.cpp
./Cpp/runtime/src/tree/xpath/XPathLexer.g4
./Cpp/runtime/src/tree/xpath/XPathLexer.h
./Cpp/runtime/src/tree/xpath/XPathLexer.tokens
./CSharp/src/Tree/Xpath/XPathLexer.cs
./CSharp/src/Tree/Xpath/XPathLexer.g4
./CSharp/src/Tree/Xpath/XPathLexer.tokens
./Java/src/org/antlr/v4/runtime/tree/xpath/XPathLexer.class
./Java/src/org/antlr/v4/runtime/tree/xpath/XPathLexer.java
./Java/target/classes/org/antlr/v4/runtime/tree/xpath/XPathLexer.class
./Python3/src/antlr4/xpath/XPathLexer.g4
./Python3/src/antlr4/xpath/XPathLexer.py
07/11-06:24:04 /c/Users/Kenne/Documents/GitHub/antlr4/runtime
$ e ./Java/src/org/antlr/v4/runtime/tree/xpath/XPathLexer.java
07/11-06:24:41 /c/Users/Kenne/Documents/GitHub/antlr4/runtime
$ diff ./Cpp/runtime/src/tree/xpath/XPathLexer.g4 ./CSharp/src/Tree/Xpath/XPathLexer.g4
3c3
< tokens { TOKEN_REF, RULE_REF }
---
> tokens { TokenRef, RuleRef }
15,17c15,17
< word: TOKEN_REF
<       |       RULE_REF
<       |       STRING
---
> word: TokenRef
>       |       RuleRef
>       |       String
22,25c22,25
< ANYWHERE : '//' ;
< ROOT   : '/' ;
< WILDCARD : '*' ;
< BANG   : '!' ;
---
> Anywhere : '//' ;
> Root   : '/' ;
> Wildcard : '*' ;
> Bang   : '!' ;
29,32c29,33
<                               if (isupper(getText()[0]))
<                                 setType(TOKEN_REF);
<                               else
<                                 setType(RULE_REF);
---
>                               String text = Text;
>                               if ( Char.IsUpper(text[0]) )
>                                       Type = TokenRef;
>                               else
>                                       Type = RuleRef;
58,59c59,60
<             |   '\uFDF0'..'\uFFFF' // implicitly includes ['\u10000-'\uEFFFF]
<             ;
---
>             |   '\uFDF0'..'\uFFFD'
>             ; // ignores | ['\u10000-'\uEFFFF] ;
61c62
< STRING : '\'' .*? '\'';
---
> String : '\'' .*? '\'' ;
63c64
< //WS : [ \t\r\n]+ -> skip ;
---
> //Ws : [ \t\r\n]+ -> skip ;
07/11-06:25:10 /c/Users/Kenne/Documents/GitHub/antlr4/runtime
$ grep -i -e generated `find . -name 'XPathLexer.*'`
./Cpp/runtime/src/tree/xpath/XPathLexer.cpp:// Generated from XPathLexer.g4 by ANTLR 4.13.0
./Cpp/runtime/src/tree/xpath/XPathLexer.h:// Generated from XPathLexer.g4 by ANTLR 4.13.0
./CSharp/src/Tree/Xpath/XPathLexer.cs:// <auto-generated>
./CSharp/src/Tree/Xpath/XPathLexer.cs://     This code was generated by a tool.
./CSharp/src/Tree/Xpath/XPathLexer.cs://     the code is regenerated.
./CSharp/src/Tree/Xpath/XPathLexer.cs:// </auto-generated>
./CSharp/src/Tree/Xpath/XPathLexer.cs:// Generated from XPathLexer.g4 by ANTLR 4.11.2-SNAPSHOT
./CSharp/src/Tree/Xpath/XPathLexer.cs:[System.CodeDom.Compiler.GeneratedCode("ANTLR", "4.11.2-SNAPSHOT")]
./Python3/src/antlr4/xpath/XPathLexer.py:# Generated from XPathLexer.g4 by ANTLR 4.11.2-SNAPSHOT
07/11-06:27:00 /c/Users/Kenne/Documents/GitHub/antlr4/runtime
$

Note: For one of the current ongoing rewrites of Antlr, I'd highly recommend a complete rewrite of the Antlr tree representation in order to support tree edits and querying of off-channel content, along with replacing the current XPath engine in the with a real one, preferably Selenium, which is the gold standard. Trash uses a port of the ancient Xerces engine, which is excellent, but only supports XPath version 2. I'm still porting Selenium to C#.

from antlr4.

jimidle avatar jimidle commented on August 15, 2024

A difference in case. Not sure why there is such a difference, but it I can't see how this is an issue. I've never had to write a target agnostic frontend/compiler in the last 3 decades... er, ever.

Also, ANTLR produces parse trees. They are only useful for facilitating production of a more formal AST. That's where we do the micro optimizations. If writing a real world compiler, generate LLVM and leave it there.

This is not an official position in any way. Just my thoughts. Python for parsing? Hmmm

from antlr4.

kaby76 avatar kaby76 commented on August 15, 2024

A difference in case.

'\uFDF0'..'\uFFFF' is not the same range as '\uFDF0'..'\uFFFD'. It is not a matter of a difference in case.

from antlr4.

jimidle avatar jimidle commented on August 15, 2024

from antlr4.

jules-l-jimmy avatar jules-l-jimmy commented on August 15, 2024

Any updates on this ? Or is there an option to hide the test, in the source, it seems that it cannot be skipped ?

from antlr4.

parrt avatar parrt commented on August 15, 2024

we don't always regen those but I'm gonna do a release soon and will try to regen those.

from antlr4.

Thomasb81 avatar Thomasb81 commented on August 15, 2024

Should we propose PR to remove this print/check from python runtime ?
It seems not there in other runtime.

from antlr4.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.