Comments (3)
Can you post a spec file with input that doesn't work for you as expected? The rules above are not in the JFlex syntax.
I'd expect the following to work:
%%
%public
%class Test
%debug
letters = [a-zA-Z]+
end = "stop"
%%
{letters} / {end} { return 0; }
[^] { return 1; }
For input
somethingstop
you should get a single match for something
, returning 0
, and single matches afterwards for s
, t
, o
, p
.
from jflex.
Sorry for being much too short.
Your example can be compiled.
Taking this as a starting point, I tried to use trailing-context in a macro, see the example below. This is not supported and results in the described syntax error.
%%
%public
%class Test
%debug
letters = [a-zA-Z]+
end = "stop"
stopped = {letters} / {end}
%%
stopped { return 0; }
[^] { return 1; }
It seems that, as a special case, trailing context regular expressions cannot be used in macro definitions.
I think that it would make it simpler to define complex lexical specs if that would be no special case, so I propose that as a new feature for JFlex: Offer support for trailing context rules in macros.
Background: The lexer I'm working on can be seen here. This implements the lexical structure for the Scheme programming language v5. So far, it was possible to define all scanning expressions as macros, matching only the terminal tokens in the match section. This is now not longer the case.
The requirement for using trailing context came in with the new Scheme v7 feature of nested comments, I added the relevant part of the lexical structure spec (note the comment text rule):
<comment> −→ ; all subsequent characters up to a line ending
| <nested comment>
| #; <intertoken space> <datum>
<nested comment> −→ #| <comment text>
<comment cont>* |#
<comment text> −→ <character sequence not containing
#| or |#>
<comment cont> −→ <nested comment> <comment text>
from jflex.
It seems that, as a special case, trailing context regular expressions cannot be used in macro definitions.
That is indeed the case, and error reporting for it is not exactly great. I've opened #1089 for that.
I think that it would make it simpler to define complex lexical specs if that would be no special case, so I propose that as a new feature for JFlex: Offer support for trailing context rules in macros.
This is going to be tricky, and I'm not sure yet it's worth it. The underlying problem is that the matching algorithm only works for trailing context at the top level (otherwise we're not matching a regular language any more). If we allow trailing contexts in macros, we can now syntactically get trailing context at any nesting level in an expression.
I guess we could allow it syntactically and have a semantic pass for checking that it is only used at the top level.
from jflex.
Related Issues (20)
- UnicodeCaseless.flex.vm uses incorrect unicode escapes HOT 1
- add test cases for warn/no-warn options
- optimisation: factor out common prefixes in large alternatives
- provide mechanism for annotations on lexer function
- investigate Windows test runners for CI
- matrix for bazel build
- Whitespaces negation in group not working as expected HOT 4
- jflex-maven-plugin 1.9.0/1 are missing from maven central HOT 1
- Generation aborted HOT 1
- improve error reporting for lookahead in macros HOT 2
- potential ArrayIndexOutOfBounds in zzRefill
- generated-class.md missing from manual published on website HOT 4
- Is there a way to suppress comment/absolute path in the generated code? HOT 6
- Unexpected dump? HOT 7
- Unexpected exception encountered in JFlex HOT 3
- char class directives delete existing classes HOT 1
- replace inefficient java regexps with jflex scanner HOT 1
- properly implement clone interface on JFlexAntTask
- allow length-1 expressions in char classes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jflex.