Comments (5)
Is there an ANTLR bug involved? This appears to be a unit test for another application which uses ANTLR but isn't testing the ANTLR library itself. The expectedTokens
test doesn't contain any assertions, so it's unclear what the intent actually is.
from antlr4.
Tried to clarify the code a bit, please re-check. The test concerned is expectedTokens().
Unfortunately, the stuff lies in the output to stdout. I spit out what I have parsed so far and then what tokens are to be expected using Parser#getExpectedTokens(). All I see as the expected token is the token I just parsed.
Maybe I just do not grok how this is supposed to work.
from antlr4.
Parser.getExpectedTokens()
uses the current state of the current context (parser._ctx.s
) determine the set of expected tokens. As soon as the parser changes state, the set of expected tokens changes. Right before a call to match
(which produces the call to visitTerminal
), the state is set to the source state of the non-epsilon transition matching the symbol. If you call getExpectedTokens()
from within visitTerminal
of a parser listener, you'll get the set of tokens for the single transition that matched the terminal (which is exactly what you saw).
from antlr4.
Hey, Sam, thanks for being patient with me. I actually mixed up stuff here and what you say makes sense. For I actually want to do: Finding out the set of valid tokens that can follow on a given position inside a buffer. Like for an editor with auto-completion. Any pointers?
from antlr4.
In ANTLRWorks 2 (currently closed source due to licensing details I'm working on), to get this information I needed to extend ParserATNSimulator
, CommonToken
, ANTLRErrorStrategy
, TokenSource
, and use a special RuntimeException
to bail out of the internals while preserving the interpreter state, as well as controlling the beast with a special controller. In addition, I had to add the -Xforce-atn
option to the tool (and use it for any parser where I intend to derive this information).
It's gruesome, but what's possible here is without a doubt the most powerful solution to this problem I've ever seen. Proof of this should be clear in what I was able to accomplish in GoWorks in under 1 month.
from antlr4.
Related Issues (20)
- Use the new version antlr4-python3-runtime 4.13.1 import latex2sympy2 error HOT 1
- Lexing Issue in ANTLR4 Grammar for Fortran 2018: Token Misclassification HOT 1
- Windows CPP Symbol Conflicts (EMPTY, WIN16, WIN32, WIN64)
- Is this project out of date?
- ANTLR doesn't generate needed methods for C++ HOT 1
- Add methods for Go runtime
- Add some modifications for Dart runtime
- [antlr4-python3-runtime-4.13.1] ANTLR runtime and generated code versions disagree: 4.13.1!=4.11.2-SNAPSHOT HOT 14
- [Cpp] XPath.findAll does not produce the same result than python3 runtime
- Easy to reproduce baffling parse failure when using semantic predicates. Bug in parser generator? HOT 9
- [TypeScript]: The module βantlr4β or the associated type declarations were not found.ts(2307) HOT 2
- Go runtime should not depend on Go 1.22 yet HOT 4
- Go target: lexer constants with the private access if g4 filename starts with the lowercase
- [Feature request] Support range operator for tokens HOT 1
- Incorrect Handling of Syntax Errors in MySqlParser with Custom Error Strategy
- javascript runtime v4.13.1 not found api: getVocabulary()
- How to make antlr4 skip the intermediate errors and continue parsing backward when parsing Sql? HOT 1
- generate go lexer file error HOT 2
- Always error: line 1:0 mismatched input 'Token' expecting {<EOF>,Token} HOT 5
- ruleIndex property missing in ParserRuleContext itypescript type
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from antlr4.