dryruby / ebnf Goto Github PK

I'd love to see support for ISO/IEC 14977 1996(E).

str = "+#{'0' * 10000}\n+"
s = EBNF::LL1::Scanner.new(StringIO.new(str))
s.scan(/\+.*\+/m)
# => nil

Increasing both :low_water and :high_water to a size larger than the token fixes this.

s = EBNF::LL1::Scanner.new(StringIO.new(str), :low_water => 20000, :high_water => 20000)
s.scan(/\+.*\+/m)

When increasing only :high_water, there's a possibility that the previous scan/feed will leave us with a #rest containing only part of the next token.

I'm not sure of a way to fix this, beyond giving up on low/high water scanning altogether; it seems like this would be a problem for any language with arbitrarily long multiline terminals.

Describe the bug
Parsing ABNF files does not work.

To reproduce
The following uses the latest version of the library and opens examples/abnf/examples/postal-address.abnf from this repo.

> pry
[1] pry(main)> require 'ebnf'
=> true
[2] pry(main)> grammar = EBNF.parse(File.open('postal-address.abnf', format: :abnf))
SyntaxError: ERROR [line: 1] syntax error, expecting "@terminals", "@pass", :LHS (found "postal-address   = n")
from /Users/aaronlasseigne/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/ebnf-2.3.0/lib/ebnf/parser.rb:302:in `rescue in initialize'
Caused by EBNF::PEG::Parser::Error: ERROR [line: 1] syntax error, expecting "@terminals", "@pass", :LHS (found "postal-address   = n")
from /Users/aaronlasseigne/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/ebnf-2.3.0/lib/ebnf/peg/parser.rb:253:in `parse'

Expected behavior
Parsing ABNF files works.

Describe the bug
I think the example links may need updating

To reproduce
Steps to reproduce the behavior:

1. Click on any example link in README or on docs website
2. See 404 error

Expected behavior
HTTP 200

Screenshots

Desktop (please complete the following information):
N/A

Smartphone (please complete the following information):
N/A

Additional context
N/A

Description
I get SyntaxError in some rules as simple as letter ::= [A-Z]. It seems to be caused by another rule being just after it, but it does not seem that simple. I attach a program to show the cases I checked.
The error produced is like this:

$ irb --nocolorize --simple-prompt -r ebnf
>> EBNF.parse("letter ::= [A]\nx ::= 'X'")
Traceback (most recent call last):
       10: from /usr/local/bin/irb:23:in `<main>'
        9: from /usr/local/bin/irb:23:in `load'
        8: from /var/lib/gems/2.7.0/gems/irb-1.2.7/exe/irb:11:in `<top (required)>'
        7: from (irb):1
        6: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf.rb:26:in `parse'
        5: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf.rb:26:in `new'
        4: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf/base.rb:124:in `initialize'
        3: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf/base.rb:124:in `new'
        2: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf/parser.rb:265:in `initialize'
        1: from /var/lib/gems/2.7.0/gems/ebnf-2.1.1/lib/ebnf/parser.rb:302:in `rescue in initialize'
SyntaxError (ERROR [line: 1] syntax error, expecting :HEX, :SYMBOL, :O_RANGE, :RANGE, :STRING1, :STRING2, "(", :primary, :postfix, :seq, :alt, :expression (found "[A]\nx ::= 'X'"))
>> 

To reproduce
Run the attached ebnf_debug.rb.txt to see examples and counterexamples.
§ denotes newline as you can check in the program.
For me it prints:

$ ruby ebnf_debug.rb 
Ruby: 2.7.0,  ebnf: 2.1.1
letter ::= [A-Z]                          : OK
letter ::= [A]                            : OK
digit ::= [0-9]                           : OK
letter ::= [A-Z] §                        : OK
letter ::= [A-Y] | 'Z'                    : OK
word ::= [A-Z]+                           : OK
letter ::= [A-Z] /* x */                  : OK
letter ::= [A-Z]       § digit ::= [0-9]  : SyntaxError
digit ::= [0-9]        § letter ::= [A-Z] : SyntaxError
letter ::= [A-Z]       § word ::= letter+ : SyntaxError
letter ::= [A-Z]       § x ::= 'X'        : SyntaxError
letter ::= [A]         § x ::= 'X'        : SyntaxError
letter ::= [A] | 'B'   § x ::= 'X'        : OK
letter ::= 'B' | [A]   § x ::= 'X'        : SyntaxError
letter ::= [A] | [B]   § x ::= 'X'        : SyntaxError
letter ::= [A] x       § x ::= 'X'        : OK
letter ::= [A-Z] x     § x ::= 'X'        : OK
letter ::= x [A]       § x ::= 'X'        : OK
letter ::= x [A-Z]     § x ::= 'X'        : SyntaxError
letter ::= [A] /* x */ § x ::= 'X'        : OK
letter ::= [A-Z] /* */ § x ::= 'X'        : OK
letter ::= [^A]        § x ::= 'X'        : OK
                                          : OK

Expected behavior
A successful parse of each above indicated example.

Desktop (please complete the following information):

• OS: Ubuntu 20.04.1 LTS
• Browser: (?) not applicable

I was wondering why my project was pulling in the HAML stack (a bit of a WTF moment), and eventually figured out that it's being pulled in transitively through rdf-turtle to ebnf. It does seem perhaps a tad heavy-handed to pull in HAML to just parse Turtle--any chance of making that a development dependency instead of a runtime one?

Create complete parser examples, for example an EBNF parser bases on the EBNF grammar itself.

Is your feature request related to a problem? Please describe.
When multiple grammars are included in the same HTML page, the rule anchors can overlap.

Describe the solution you'd like
Overload the --prefix option for adding a prefix to generated anchors.

Hi,
I wanted to play around with a simple grammar but I'm running into problems, even when using the example ebnf: ebnf = EBNF.parse(File.open('tmp/ebnf.ebnf'))
1.

ebnf.to_ebnf
NoMethodError: undefined method `to_ebnf'

ebnf.to_ruby
NoMethodError: undefined method `sort_by' for nil:NilClass
	from /var/lib/gems/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1.rb:270:in `outputTable'

Thanks!

It is referenced in the README, but grepping the code doesn't find it anywhere.

fwiw, I'm trying to parse YAML using a simple EBNF, but have no idea what to do after I've called EBNF.parse. I can output using to_sxp, but it's not clear to me how to link this to a parser class. The example parser doesn't appear to actually be driven by a bnf file?