jflex-de / jflex Goto Github PK

Reported by lsf37 on 2001-02-28 13:31 UTC
A new regular expression <<EOF>> matching the end of
file.

This feature request was made by Fabien Coelho.

Reported by lsf37 on 2001-02-28 13:36 UTC
Negated character class expressions that contains the
last character of the input character set (in the text
of the expression) may cause an exception in JFlex at
generation time.

Example: with the 8 bit input character set, the
expression [^\xFF] will cause an exception, while
[^a-z] or [^\x00] will not.

This bug was reported by Al George.

(fixed in 1.2)

Reported by lsf37 on 2001-02-28 14:00 UTC
If two lexical states are equivalent (i.e. are used
only for the exact same actions), and if they are used
in an <<EOF>> rule,the generated code will not compile.

Workaround: remove one of the redundant states.

This bug was reported by Alexey Yakovets.

(fixed in 1.3.1)

Reported by lsf37 on 2001-02-28 13:54 UTC
Make the size of the scanner's readahead buffer
configurable from specifications.

This feature was requested by Hans Kratz.

Reported by lsf37 on 2001-02-28 11:30 UTC
In some cases (i.e. for some specifications) the %pack
directive produces scanner code that matches wrong
portions of text (i.e. too much or too few). The other
code generation methods are not affected.

This bug was reported by Frank Berger.

(fixed in 1.1.1)

Reported by lsf37 on 2001-03-02 08:49 UTC
If the last thing in an action code block of a rule is an end of line comment (// comment) the
generated code will not compile.

Workaround: use a /* ... */ style comment.

This bug was reported by Ole Engel.

(fixed in 1.2.1)

Reported by lsf37 on 2001-03-10 17:56 UTC
javac allows // comments at the end of file (without a line terminator). The java lexer
example that comes with JFlex requires a line terminator. Although the Java lang spec doesn't
contain this, it's easy to add and doesn't affect the rest of the language.

reported by Thomas Mulcahy

Reported by lsf37 on 2001-09-27 12:47 UTC
the exceptions declared with %yylexthrow are ignored in
the main method generated for %standalone scanners. The
generated code does not compile.

This bug was reported by Mike Akerman. It will be fixed
in the next release.

Reported by lsf37 on 2001-02-28 11:36 UTC
Documentation of lex/flex compatibility, i.e. how to
port lex specifications to JFlex
specifications.Compatibility to lex/flex should be
enhanced.

This feature request was made by Hĺkan Söderström.

(done some of it for version 1.2)

Reported by lsf37 on 2001-09-27 12:51 UTC
The example scanner in the manual is missing some
backslashes.

This bug was reported by Angelo Borsotti and Paolo Di
Francesco.

Reported by lsf37 on 2001-09-27 12:54 UTC
The CUP symbol class should be made customizable (for
debugging and EOF value return).

This bug was reported by Karl Meissner and Paolo Di
Francesco.

Reported by lsf37 on 2001-02-28 13:52 UTC
Customization for internal error method code.

This feature was requested by Sean Behiel.

(done in 1.3)

Reported by lsf37 on 2001-02-28 13:43 UTC
JFlex produces an incorrect scanner if the %full,
%8bit, %unicode or %16bit switches are used after one
or more macros have already been declared.

Workaround: use these switches before you declare any
macros.

This bug was reported by Binesh Bannerjee.

(fixed in 1.2.1)

Reported by lsf37 on 2001-02-28 11:24 UTC
The %eofval directive did override %cup settings in
version 1.0 of JFlex. Version 1.1 does not allow this
(produces not compilable code). Release 1.1.1 of JFlex
now contains the (initially not intended but very
useful) override feature with proper documentation in
the manual.

This feature request was made by Duncan Temple Lang.

Reported by lsf37 on 2001-10-03 12:02 UTC
JFlex doesn't recognize characters escaped by ".."
inside character classes.

Example: ["ab"] will be parsed as the class that
contains the three characters " a b
JLex would parse it as the class that contains a and b

This bug was reported by Amin Ahmad.

It will be fixed in the next release.

Reported by lsf37 on 2001-02-28 13:42 UTC
The generated scanners will fail when they need to
refill the internal input buffer while currently
reading a lookahead expression. The standard size of
the internal input buffer in JFlex is 16384 characters.
Inputs with fewer characters are scanned correctly.
Specifications that don't use lookahead expressions are
not affected.

This bug was reported by Wolfgang Fiedler.

(fixed in 1.2.1)

Reported by lsf37 on 2001-02-28 13:57 UTC
The new error reporting is a bit too eager: on Windows
and Mac platforms, JFlex doesn't recognize the %buffer,
%initthrow, %eofthrow, and %yylexthrow directives.
Reason: the \r line terminator is not parsed correctly
for these directives.

Workaround: Use the alternative form of the directive
for %initthrow, %eofthrow, and %yylexthrow, or set your
editor to use Unix \n line feeds.

This bug was reported by Hans Kratz and Thilo Goetz.

(fixed in 1.3.1)

Reported by lsf37 on 2001-02-28 13:45 UTC
The rules

"ab" { System.out.print("AB"); }
"a"/"b" { System.out.print("A"); }

give you with input ab the output ABb and not the
correct AB. If the rules are in reversed order
("a"/"b" and then "ab") it works ok.

This bug was reported by Tibi.

(fixed in 1.3)

Reported by lsf37 on 2001-02-28 13:52 UTC
Customization for internal error method code.

This feature was requested by Sean Behiel.

(done in 1.3)

Reported by lsf37 on 2001-07-13 09:59 UTC
The %type directive only takes qualified identifiers.
This should be changed to type as defined in the JLS
§4.1.

This bug was submitted by Axel Schwolow.

Reported by lsf37 on 2001-02-28 11:21 UTC
Incorrect parsing, if there is no blank line after the
second %%. If there is no whitespace character before
the first regular expression and after the newline of
the second %%-line and if the first regular expression
doesn't start with a lexical state condition, then
JFlex will incorrectly complain about an illegal
character in that line (and then stop) or will parse
the first regular expression incorrectly.

This bug was reported by Frank Berger.

(fixed in 1.1)

Reported by lsf37 on 2001-09-27 12:49 UTC
The usage message of JFlex doesn't include the new
1.3.3 options.

This bug was reported by Mike Akerman. It will be fixed
in the next release

Reported by lsf37 on 2001-02-28 13:59 UTC
The manual incorrectly shows

<<EOF>> { if (yymoreStreams()) yypopStream();
else return EOF; }

as the way to pop nested input streams on end of file.
This doesn't work, because JFlex expects EOF-actions
always to return a value. The error only leads to
faulty scanner behaviour when there is more than one
EOF-action (i.e. in multiple lexical states) and the
erroneous action is not the default action. That means,
if the line above is the only EOF-action in a
specification, the scanner works as expected.

Workaround: insert break statements where no value is
returned

<<EOF>> { if (yymoreStreams()) { yypopStream();
break; } else return EOF; }

This bug was reported by Vasily Karyaev.

(fixed in 1.3.1)

Reported by lsf37 on 2001-02-28 13:52 UTC
A method char yycharat(int pos) for efficient access to
single characters in matched text.

This feature was requested by Tatu Saloranta.

(done in 1.3)

Reported by lsf37 on 2001-02-28 13:34 UTC
Due to a bug in the JIT implementation of JDK 1.1.6
under Solaris, JFlex does not work properly under
Solaris with JDK1.1.6. This is also true for very large
scanners generated by JFlex.

Workaround: switch off the JIT compiler temporarily

JFlex works smoothly on Solaris/Sparc with JDK 1.1.3
(positively tested) and Solaris/Sparc JDK 1.2 and
above. See also the list of platforms JFlex has been
positively tested on.

This bug was reported by Luca Iocchi. It is not really
fixed in the sense, that the bug is now not present in
the Solrais JIT compiler, but JFlex scanners do not
trigger the JIT bug any more.

Reported by lsf37 on 2001-05-04 21:15 UTC
for some lookahead expressions JFlex will fail to print a warning that they are dangerous (r1 / r2 is
dangerous if any postfix of r1 matches a prefix r2).

Example:

.+ "a" / "b" .+ { action }

will not produce a warning/error message, but the postfix .+"a" of r1 matches a prefix of r2 (the
whole lookahead).

These lookahead expressions might match some inputs incorrectly. It is guaranteed that the rule
gets chosen if there is a match for the expression r1r2 (concatenation of r1 and r2), and it is
guaranteed that yytext() matches r1. It is not guaranteed that there is an instance of r2 after
yytext() (because too few lookahead characters were put back into the input).

Workaround: check manually if your lookahead expressions are dangerous.

(reported by Robert Hubley)

Reported by lsf37 on 2001-02-28 11:34 UTC
The %type directive accepts only simple identifiers as
parameter (as for instance just Symbol). This should
be changed to qualified identifiers (as for instance
java_cup.Symbol).

This bug was reported by Fei Sha.

(fixed in 1.1.2)

*Reported by anonymous on 2001-12-10 11:09 UTC
When referring to an undeclared definition in a
complement expression no error is reported:

%%

r = !{s}

%%

will be accepted, but a NullPointerException is thrown
when the dfa is constructed as `s' is not defined.

Reported by lsf37 on 2001-02-28 13:29 UTC
In debug mode (in specifications with the %debug
directive) the generated main method should catch the
exceptions declared in the %yylexthrow directive.

This bug was reported by Hĺkan Söderström.

Reported by lsf37 on 2001-02-28 14:02 UTC
JFlex may incorrectly report a lookahead expression as
illegal if the first character of the lookahead is
described by a negated character class and the first
part of the expression does not have a fixed length.
The checking algorithm ignores the negation in this
case. Only the error checking algorithm is effected,
code generation works correctly. Example: the
expression [a]+/[^a] is a legal lookahead expression,
but rejected.

Workaround: describe the negated character class in
positive form.

This bug was reported by Tom Moog.

(fixed in 1.3.1)

Reported by lsf37 on 2001-09-27 12:56 UTC
when the upto operator is applied to a RE which
generates the
empty string, its definition ( !([^]* a [^]*) a ) would
produce the empty language, which is not what is
expected. I.e. "any string up to the first occurrence
of an empty string" should be an empty string.

I guess that there is a need to include an alternative
for the empty string:
( !([^]* a [^]*) | "" ) a

This bug was reported by Angelo Borsotti

Reported by lsf37 on 2001-02-28 11:12 UTC
Minor incompatibility between JFlex and JLex in the
rules section of the specification. JFlex needs a
whitespace character between the trailing > of the
lexical states and the start of the regular expression:

JLex lets you write
<YYINITIAL>\{DISCHARGE\} \{...\}
and JFlex needs
<YYINITIAL> \{DISCHARGE\} \{...\}

Original report by Alfred Waas, fixed in 1.1

Reported by lsf37 on 2001-02-28 13:48 UTC
The %yylexthrow directive ignores the first letter of
its parameter.

Workaround: insert one more space character between
%yylexthrow and the exception. Instead of

%yylexthrow exception

write

%yylexthrow  exception

This bug was reported by Frank Berger.

(fixed in1.3)

Reported by lsf37 on 2001-02-28 13:50 UTC
Unicode escapes in character literals like '\u000A' in
action code are not recognized correctly. Only
character literals are affected, standard escape
sequences like '\n' and all escape sequences in strings
work ok.

This bug was reported by Kai Knubben.

(fixed in 1.3)

Reported by lsf37 on 2001-02-28 14:05 UTC
If a macro itself references an undefined macro in the
first part of a lookahead expression, or in the second
part of a lookahead expression when the first part
doesn't have fixed length, then JFlex will give a stack
trace instead of a controlled error message.

Workaround: don't use undefined macros

This bug was reported by Tom Moog.

(fixed in 1.3.1)

Reported by lsf37 on 2001-02-28 13:55 UTC
On Solaris 7/Intel with JDK 1.2.2_05a, the generated
code does not compile if JFlex is run with the JIT
compiler turned on.
It seems to be a problem in the JIT compiler sunwjit
for this platform.

Workaround: switch off the JIT compiler for the JFlex
run.

This bug was reported by Hĺkan Söderström

Reported by lsf37 on 2001-02-28 11:32 UTC
A repeat statement r{n} will repeat the expression n+1
times instead of n times (the other, more general type
of repeat expressions works as specified).

This bug was reported by Petter Edblom.

(fixed in 1.1.2)

Reported by lsf37 on 2001-02-28 11:18 UTC
Another minor incompatibility between JFlex and JLex:
The java.io.Reader in generated Lexers of JLex is
called yy_reader. In JFlex it has the name yy_in.

This bug was reported by Hendrik Schreiber.
(fixed in 1.1)

Reported by lsf37 on 2001-10-04 11:15 UTC
Under some circumstances predefined character classes
like [:jletter:] don't work correctly, [:jletter:] may
for instance be parsed as the class that contains the
characters j l e t r :

The circumstances: If the predefined character class is
preceded by whitespace and not at the beginning of a
regular expression it will be parsed incorrectly.
Otherwise it will be parsed correctly.

Workaround: remove whitespace in front of predifined
character classes or use the [[:jletter:]] form instead
of [:jletter:].

This bug was reported by Fred Wehrhahn.

It will be fixed in the next release of JFlex.

Reported by lsf37 on 2001-03-02 08:44 UTC
In generated scanners the yy_close() method closes the input stream but doesn't reset the
internal scan buffer. If the buffer is not empty, the scanner will not stop immediatly (as it should)
but continue to read charcters from the buffer.

This bug was reported by Stanislav Pavlicek.

(fixed in 1.2)

Reported by lsf37 on 2001-02-28 13:49 UTC
Possible NullPointerException when reading an external
skeleton file.

This bug was reported by Hans Kratz.

(fixed in 1.3)

Reported by lsf37 on 2001-02-28 11:16 UTC
In rare cases, a stack overflow can occur during the
minimization of the DFA. The bug has up to now only
occured with large and redundant DFAs (more than 700
states and more than 200 of them redundant). This bug
may also be experienced as an endless loop, if you have
much memory available on your machine.

(fixed in 1.1)

Reported by michab66 on 2001-05-02 12:45 UTC
Hi,

I'm just in the process of evaluating scanners for a
small project I have. For JFlex I scanned through the
docs and what I saw is impressive. Good work!

The thing I was missing from the pdf document is that
it doesn't describe the generated classes. How does
the model look like to this end? This is important
for me since JFlex will have to replace an existing
scanner.

A short chapter describing this would help other
reviewers.

Keep up the good work and thanks,
Michael.

Reported by lsf37 on 2001-02-28 11:31 UTC
As soon as the %pack bug of version 1.1 is fixed, now
version 1.1.1 has a problem with the default code
generation method for some specifications.

This bug was reported by Petter Edblom.

(fixed in 1.1.2)

Reported by lsf37 on 2001-02-28 11:22 UTC
ArrayIndexOutOfBoundsException while converting the NFA
to a DFA, if the specification declares more than 25
lexical states.

This bug was reported by Ravi Ramamirtham.
(fixed in 1.1)

Reported by lsf37 on 2001-09-27 12:59 UTC
The bash script to start JFlex doen't correctly find
the installation location and therefore stops with an
error message.

Workaround: set JFLEX_HOME manually

This bug was reported by Mike Akerman.

Reported by lsf37 on 2001-02-28 13:47 UTC
The internal error handling method of generated
scanners contains calls to System.out.println and
System.exit. Although this method will (should) never
be called in an usual scanner, it may still be a
problem for some systems (i.e. applets).

This bug was reported by Sean Behiel.

Reported by lsf37 on 2001-02-28 13:30 UTC
Incompatibility to JLex: the character class expression
[+-] is legal in JLex (but not documented in the JLex
manual) - for JFlex you need an explicit escape: [+-]

This bug was reported by Kirk Abbott.

(fixed in 1.2)

Reported by lsf37 on 2001-02-28 13:52 UTC
Customization for internal error method code.

This feature was requested by Sean Behiel.

(done in 1.3)

Reported by lsf37 on 2001-02-28 13:40 UTC
Some bugs in the Java parser example (in examples/java)
concerning large numerical literals.

These bugs were reported by Wolfgang Fiedler.

(fixed in 1.2.1)