katef / kgt Goto Github PK

View Code? Open in Web Editor NEW

558.0 19.0 29.0 1.31 MB

BNF wrangling and railroad diagrams

License: BSD 2-Clause "Simplified" License

Makefile 4.61% CSS 0.19% XSLT 0.85% C 94.35%

bnf grammar parsing syntax ebnf wsn abnf cfgs parser parser-library

kgt's People

Contributors

Stargazers

Watchers

kgt's Issues

Incorrect railroad diagram

$ cat ./eg.ebnf
binding-names = {name, ","}, name, ",", name, [","];
$ ./build/bin/kgt -l iso-ebnf -e rrutf8 < ./eg.ebnf
binding-names:
                      ╭────>────╮
                      │         │
    │├──╭── name ──╮──╯── "," ──╰──┤│
        │          │
        ╰── "," ───╯

However this diagram isn't accurate, as it permits just "name", which isn't allowed by the grammar. The correct output would look like

binding-names:
                                     ╭────>────╮
                                     │         │
    │├──╭── name ──╮── "," ── name ──╯── "," ──╰──┤│
        │          │
        ╰── "," ───╯

This may be related to #61 as they both have very similar inputs.

Support for binary strings

ABNF:

x = %x00-FF

Literals can't be null-terminated strings. Possibly binary content could be a separate type, but I'd prefer just one type for all kinds of literals.

Segfault in `collapse_suffix`

With the following input:

binding-names = {name, ","}, name, ",", name
        ;

kgt segfaults when producing html5

$ cat demo.ebnf 
binding-names = {name, ","}, name, ",", name
        ;
$ gdb -q ./build/bin/kgt
Reading symbols from ./build/bin/kgt...
(gdb) r -l iso-ebnf -e html5 < demo.ebnf 
Starting program: /home/nixon/hare-cross/ebnf/kgt/build/bin/kgt -l iso-ebnf -e html5 < demo.ebnf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
<!DOCTYPE html>
<html>

 <head>
  <style>
      rect, line, path { stroke-width: 1.5px; stroke: black; fill: transparent; }
      rect, line, path { stroke-linecap: square; stroke-linejoin: rounded; }
      path { fill: transparent; }
      text.literal { font-family: monospace; }
      a { fill: blue; }
      a:hover rect { fill: aliceblue; }
      h2 { font-size: inherit; font-weight: inherit; }
      line.ellipsis { stroke-dasharray: 1 3.5; }
      tspan.hex { font-family: monospace; font-size: 90%; }
      path.arrow { fill: black; }
      svg { margin-left: 30px; }
  </style>
 </head>

 <body>

Program received signal SIGSEGV, Segmentation fault.
0x00005555555715d2 in collapse_suffix (changed=0x7fffffffd980, head=0x55555558dee8, loop=0x55555558deb0) at src/rrd/pretty_affix.c:101
101             list_free_upto(head, p->node);
(gdb) bt
#0  0x00005555555715d2 in collapse_suffix (changed=0x7fffffffd980, head=0x55555558dee8, loop=0x55555558deb0) at src/rrd/pretty_affix.c:101
#1  0x0000555555571a61 in rrd_pretty_affixes (changed=0x7fffffffd980, n=0x7fffffffda30) at src/rrd/pretty_affix.c:191
#2  0x0000555555572eaf in node_walk (f=0x555555571979 <rrd_pretty_affixes>, changed=0x7fffffffd980, n=0x7fffffffda30) at src/rrd/pretty.c:27
#3  0x00005555555730c5 in rrd_pretty (rrd=0x7fffffffda30) at src/rrd/pretty.c:83
#4  0x000055555557bd6a in output (grammar=0x55555558d9a0, xml=0) at src/html5/output.c:126
#5  0x000055555557bf1e in html5_output (grammar=0x55555558d9a0) at src/html5/output.c:166
#6  0x0000555555557a04 in main (argc=0, argv=0x7fffffffdd60) at src/main.c:270
(gdb) p p
$1 = (struct list *) 0x0

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

fputs ⇒ cat
printf ⇒ output_term

Compilation instructions.

Running on Debian Unstable, GNU Make 4.3, I get this:

mdione@diablo:~/src/projects/kartozan/take2/kgt$ make
Makefile:40: *** falta un separador.  Alto.

(sorry for the Spanish :) it says 'missing separator. stop'. Seems like the Makefile is in BSD format? Could you include at least a brief description of the dependencies and compilation instructions?

Makefile for man/kgt.1/kgt.1

I’m trying to build the manpage, but I can’t find a Makefile rule for building the manpage and/or documentation from the minidocbook description.

Am I missing something or is it just not configured yet?

segfault for ABNF range production

The following ABNF rule:

obs-text = %x80-FF

causes a segmentation fault (Ubuntu-on-Windows).

ABNF: empty line required between productions

For this input:

one = "one"
two = "two"

the following error is reported:

2:5: Syntax error: expected production rule separator

Build fails with recent bmake

When using a recent bmake, the build fails:

build flags: -j2 SHELL=/nix/store/qqa28hmysc23yy081d178jfd9a1yk8aw-bash-5.2-p15/bin/bash -r PREFIX=\$\(bin\)
bmake: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 93: Inconsistent operator for all
bmake: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 93: Inconsistent operator for clean
bmake: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/init.mk" line 90: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 93: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 93: Inconsistent operator for clean
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/init.mk" line 90: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 102: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/prog.mk" line 140: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/subdir.mk" line 102: Inconsistent operator for all
bmake[1]: "/nix/store/5im2v8qa04dq05ywv5nd5779bxf0kc3k-bmake-20230126/share/mk/final.mk" line 18: Inconsistent operator for install
bmake[1]: Fatal errors encountered -- cannot continue

Last bmake known working: 20220928. First known failing: 20230126.

https://hydra.nixos.org/build/207433254/nixlog/3

Makefile missed DIR

The Makefile under the root of kgt, An instruction missed:
SUBDIR += src

DIR += ${BUILD}/lib // <- missed

That make compile failed.
Thanks.

ABNF: silently chokes on dhall.abnf

https://github.com/dhall-lang/dhall-lang/blob/60629b9d198af49984e9f9da337036942147af8b/standard/dhall.abnf

This is the ABNF file for the dhall language, it might not be a fully spec-compliant abnf but when I feed it into kgt:

$ > kgt -l abnf -e svg <./dhall-lang/standard/dhall.abnf
[1]$

it silently exits with a status 1 without producing an error message or any output.

examples/ebnf2.ebnf not parsed as iso-ebnf

$ ./build/bin/kgt -e rrutf8 -l iso-ebnf < examples/ebnf2.ebnf
$ echo $?
1

I'm not sure if this their is a different dialect this is suposed to be, or if it needs to be moved to iso-ebnf.

Stop using exit(EXIT_FAILURE)

Hi Kate,

What do you think about this change I would like to make:
I have some trouble with the uses of:

fprintf(stderr, "unimplemented\n");
exit(EXIT_FAILURE);

This causes the program to exit, which make it unusable when kgt is used as a library. Because it doesn't return to the caller.

Instead, I would like to transform the output_XXX functions to return the error to the caller (via a bool, an enum, or an output params). Instead of calling exit, we will unwind the stack properly.
Callers would then be in a good position to decide what to do when KGT can't produce the output.

Is ABNF supported?

I downloaded and built kgt and then ran this command:

./build/bin/kgt -l abnf -e svg < ./examples/abnf.abnf > ./abnf.svg

No message and the output length of file abnf.svg is 0. However, expr.bnf to expr.svg works fine. Have not tried other flavours yet.

Examples about usage statements

(1)

There is no examples of usage statements . By using trial and error , I could find the following statements :

[s@localhost kgt]$ Installed/bin/kgt -l bnf -e dot < examples/postal.bnf > postal.dot
[s@localhost kgt]$ Installed/bin/kgt -l bnf -e dot < examples/expr.bnf > expr.dot
[s@localhost kgt]$ Installed/bin/kgt -l abnf -e dot < examples/utf8.abnf > utf8.dot

The above statements are produced viewable .dot files .

[s@localhost kgt]$ Installed/bin/kgt -l iso-ebnf -e dot < examples/json.iso-ebnf > json.dot
25:33: Syntax error

Additional usage examples may be useful .

(2)

In README

pmake

is not available in every OS .

There is "bmake" to distinguish between GNU make and BSD make .

Mentioning bmake in README may be useful .

[s@localhost kgt]$ uname -a
Linux localhost.localdomain 4.18.11-200.fc28.x86_64 #1 SMP Sun Sep 30 15:31:40 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Fedora 28

(3)

In main.c :

in xusage :

Addition/modification of the following lines may be useful :

printf("usage: kgt [-nu] [-l <language>] [ -e <language> ]   <  input_file_name  >   output_file_name   \n");

printf(" -n : prettify
printf(" -u : allow undefined
printf(" -l : input language ( see io[]= in main.c for valid input names enclosed within ""
printf(" They are : bnf , wsn , abnf , iso-ebnf , rbnf
printf(" -e : output language ( see io[]= in main.c for valid input names enclosed within ""
printf(" They are : bnf , wsn , iso-ebnf , rbnf , sid , dot , rrdot , rrdump , rrparcon , rrta , rrtext
printf(" If there is a NULL in a column , do not use that name for -l or -e
printf(" h or ? : help
printf("

Explanation for "prettify" and "allow undefined" may be useful .
Links about wsn , rbnf , sid , dot , rrdot , rrdump , rrparcon , rrta , rrtext in README may be useful .

error: implicit declaration of function 'strdup' during installation

; bmake -r install                                                                                                   ⬡ 12.6.0 [±main ✓]
mkdir -p build
mkdir build/src
mkdir build/src/bnf
mkdir build/src/blab
mkdir build/src/ebnfhtml5
mkdir build/src/dot
mkdir build/src/abnf
mkdir build/src/iso-ebnf
mkdir build/src/rbnf
mkdir build/src/sid
mkdir build/src/wsn
mkdir build/src/rrd
mkdir build/src/rrdump
mkdir build/src/rrtdump
mkdir build/src/rrparcon
mkdir build/src/rrll
mkdir build/src/rrta
mkdir build/src/rrtext
mkdir build/src/rrdot
mkdir build/src/svg
mkdir build/src/html5
mkdir build/lib
mkdir build/bin
gcc -M -o build/src/bnf/lexer.mk -MT build/src/bnf/lexer.o -ansi -pedantic -D LX_HEADER='"lexer.h"' -c src/bnf/lexer.c
gcc -M -o build/src/bnf/parser.mk -MT build/src/bnf/parser.o -ansi -pedantic -D FORM='bnf' -c src/bnf/parser.c
gcc -M -o build/src/bnf/output.mk -MT build/src/bnf/output.o -ansi -pedantic  -c src/bnf/output.c
gcc -M -o build/src/blab/output.mk -MT build/src/blab/output.o -ansi -pedantic  -c src/blab/output.c
gcc -M -o build/src/ebnfhtml5/output.mk -MT build/src/ebnfhtml5/output.o -ansi -pedantic  -c src/ebnfhtml5/output.c
gcc -M -o build/src/dot/output.mk -MT build/src/dot/output.o -ansi -pedantic  -c src/dot/output.c
gcc -M -o build/src/abnf/lexer.mk -MT build/src/abnf/lexer.o -ansi -pedantic -D LX_HEADER='"lexer.h"' -c src/abnf/lexer.c
gcc -M -o build/src/abnf/parser.mk -MT build/src/abnf/parser.o -ansi -pedantic -D FORM='abnf' -c src/abnf/parser.c
gcc -M -o build/src/abnf/output.mk -MT build/src/abnf/output.o -ansi -pedantic  -c src/abnf/output.c
gcc -M -o build/src/iso-ebnf/lexer.mk -MT build/src/iso-ebnf/lexer.o -ansi -pedantic -D LX_HEADER='"lexer.h"' -c src/iso-ebnf/lexer.c
gcc -M -o build/src/iso-ebnf/parser.mk -MT build/src/iso-ebnf/parser.o -ansi -pedantic -D FORM='iso_ebnf' -c src/iso-ebnf/parser.c
gcc -M -o build/src/iso-ebnf/output.mk -MT build/src/iso-ebnf/output.o -ansi -pedantic  -c src/iso-ebnf/output.c
gcc -M -o build/src/rbnf/lexer.mk -MT build/src/rbnf/lexer.o -ansi -pedantic -D LX_HEADER='"lexer.h"' -c src/rbnf/lexer.c
gcc -M -o build/src/rbnf/parser.mk -MT build/src/rbnf/parser.o -ansi -pedantic -D FORM='rbnf' -c src/rbnf/parser.c
gcc -M -o build/src/rbnf/output.mk -MT build/src/rbnf/output.o -ansi -pedantic  -c src/rbnf/output.c
gcc -M -o build/src/sid/output.mk -MT build/src/sid/output.o -ansi -pedantic  -c src/sid/output.c
gcc -M -o build/src/wsn/lexer.mk -MT build/src/wsn/lexer.o -ansi -pedantic -D LX_HEADER='"lexer.h"' -c src/wsn/lexer.c
gcc -M -o build/src/wsn/parser.mk -MT build/src/wsn/parser.o -ansi -pedantic -D FORM='wsn' -c src/wsn/parser.c
gcc -M -o build/src/wsn/output.mk -MT build/src/wsn/output.o -ansi -pedantic  -c src/wsn/output.c
gcc -M -o build/src/rrd/node.mk -MT build/src/rrd/node.o -ansi -pedantic  -c src/rrd/node.c
gcc -M -o build/src/rrd/transform.mk -MT build/src/rrd/transform.o -ansi -pedantic  -c src/rrd/transform.c
gcc -M -o build/src/rrd/list.mk -MT build/src/rrd/list.o -ansi -pedantic  -c src/rrd/list.c
gcc -M -o build/src/rrd/tnode.mk -MT build/src/rrd/tnode.o -ansi -pedantic  -c src/rrd/tnode.c
gcc -M -o build/src/rrd/rewrite_ci.mk -MT build/src/rrd/rewrite_ci.o -ansi -pedantic  -c src/rrd/rewrite_ci.c
gcc -M -o build/src/rrd/pretty_ci.mk -MT build/src/rrd/pretty_ci.o -ansi -pedantic  -c src/rrd/pretty_ci.c
gcc -M -o build/src/rrd/pretty_collapse.mk -MT build/src/rrd/pretty_collapse.o -ansi -pedantic  -c src/rrd/pretty_collapse.c
gcc -M -o build/src/rrd/pretty_affix.mk -MT build/src/rrd/pretty_affix.o -ansi -pedantic  -c src/rrd/pretty_affix.c
gcc -M -o build/src/rrd/pretty_bottom.mk -MT build/src/rrd/pretty_bottom.o -ansi -pedantic  -c src/rrd/pretty_bottom.c
gcc -M -o build/src/rrd/pretty_nested.mk -MT build/src/rrd/pretty_nested.o -ansi -pedantic  -c src/rrd/pretty_nested.c
gcc -M -o build/src/rrd/pretty_redundant.mk -MT build/src/rrd/pretty_redundant.o -ansi -pedantic  -c src/rrd/pretty_redundant.c
gcc -M -o build/src/rrd/pretty_skippable.mk -MT build/src/rrd/pretty_skippable.o -ansi -pedantic  -c src/rrd/pretty_skippable.c
gcc -M -o build/src/rrd/pretty_roll.mk -MT build/src/rrd/pretty_roll.o -ansi -pedantic  -c src/rrd/pretty_roll.c
gcc -M -o build/src/rrd/pretty.mk -MT build/src/rrd/pretty.o -ansi -pedantic  -c src/rrd/pretty.c
gcc -M -o build/src/rrdump/output.mk -MT build/src/rrdump/output.o -ansi -pedantic  -c src/rrdump/output.c
gcc -M -o build/src/rrtdump/output.mk -MT build/src/rrtdump/output.o -ansi -pedantic  -c src/rrtdump/output.c
gcc -M -o build/src/rrparcon/output.mk -MT build/src/rrparcon/output.o -ansi -pedantic  -c src/rrparcon/output.c
gcc -M -o build/src/rrll/output.mk -MT build/src/rrll/output.o -ansi -pedantic  -c src/rrll/output.c
gcc -M -o build/src/rrta/output.mk -MT build/src/rrta/output.o -ansi -pedantic  -c src/rrta/output.c
gcc -M -o build/src/rrtext/output.mk -MT build/src/rrtext/output.o -ansi -pedantic  -c src/rrtext/output.c
gcc -M -o build/src/rrdot/output.mk -MT build/src/rrdot/output.o -ansi -pedantic  -c src/rrdot/output.c
gcc -M -o build/src/svg/output.mk -MT build/src/svg/output.o -ansi -pedantic  -c src/svg/output.c
gcc -M -o build/src/svg/path.mk -MT build/src/svg/path.o -ansi -pedantic  -c src/svg/path.c
gcc -M -o build/src/html5/output.mk -MT build/src/html5/output.o -ansi -pedantic  -c src/html5/output.c
gcc -M -o build/src/ast.mk -MT build/src/ast.o -ansi -pedantic  -c src/ast.c
gcc -M -o build/src/ast_binary.mk -MT build/src/ast_binary.o -ansi -pedantic  -c src/ast_binary.c
gcc -M -o build/src/bitmap.mk -MT build/src/bitmap.o -ansi -pedantic  -c src/bitmap.c
gcc -M -o build/src/main.mk -MT build/src/main.o -ansi -pedantic  -c src/main.c
gcc -M -o build/src/rewrite_ci.mk -MT build/src/rewrite_ci.o -ansi -pedantic  -c src/rewrite_ci.c
gcc -M -o build/src/rewrite_invisible.mk -MT build/src/rewrite_invisible.o -ansi -pedantic  -c src/rewrite_invisible.c
gcc -M -o build/src/txt.mk -MT build/src/txt.o -ansi -pedantic  -c src/txt.c
gcc -M -o build/src/xalloc.mk -MT build/src/xalloc.o -ansi -pedantic  -c src/xalloc.c
gcc -o build/src/ast.o -std=c89 -pedantic -O3 -DNDEBUG  -c src/ast.c
gcc -o build/src/ast_binary.o -std=c89 -pedantic -O3 -DNDEBUG  -c src/ast_binary.c
gcc -o build/src/bitmap.o -std=c89 -pedantic -O3 -DNDEBUG  -c src/bitmap.c
gcc -o build/src/main.o -std=c89 -pedantic -O3 -DNDEBUG  -c src/main.c
src/main.c:219:10: error: implicit declaration of function 'strdup' [-Werror,-Wimplicit-function-declaration]
                        tmp = strdup(filter);
                              ^
src/main.c:219:10: note: did you mean 'xstrdup'?
src/xalloc.h:15:7: note: 'xstrdup' declared here
char *xstrdup(const char *s);
      ^
src/main.c:219:8: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion]
                        tmp = strdup(filter);
                            ^ ~~~~~~~~~~~~~~
1 warning and 1 error generated.
*** Error code 1

Stop.
bmake: stopped in /Users/maxclaytoncowes/Development/mcclowes/kgt

Makefile, needed to create directories

Great tool. thanks very much.
"Just Worked" on MacOS 10.14 (Mojave) with clang and 'bsdmake' from Homebrew.

Only niggle is having to manually create subdirectories in BUILD dir.

build/src
build/lib
build/bin

Then all the 'SUBDIR' directories in kgt/Makefile

How can I readably display the man page - on Linux?

man and firefox don't display it usably

Exponential grow for case insensitive input.

Hi,
Thanks for this tool. This is awesome!

I noticed some generator produce all the possibles combinaisons for case incentive input. It means this generate a list of item growing exponentially relative to the input length.

For instance if you use: (input: abnf)

test = "test"

Then you get your get. (output: "rrta")

add('test', Diagram(
  Choice(0,
    Terminal("test"),
    Terminal("Test"),
    Terminal("tEst"),
    Terminal("TEst"),
    Terminal("teSt"),
    Terminal("TeSt"),
    Terminal("tESt"),
    Terminal("TESt"),
    Terminal("tesT"),
    Terminal("TesT"),
    Terminal("tEsT"),
    Terminal("TEsT"),
    Terminal("teST"),
    Terminal("TeST"),
    Terminal("tEST"),
    Terminal("TEST"))));

I was curious to know if this is something you want to keep or fix later.
If you want to fix it later, would you prefer I try to submit PR?

I tried all the generator online using this tool https://arthursonzogni.com/Diagon/#code_area

exponential output:

rrparcon
rrl
rrta

non exponential output:

rrutf8
rrtext
svg
html5
xhtml5
ebnfhtml5
ebnfxhtml5
blab
abnf
dot
rdot
rrdump
rrtdump

Can't say (reached unimplemented)

bnf
wsn
iso-ebnf
rbnf
sid

ABNF > Blab - Core rules not exported

When generating blab output from ABNF that uses built-in core rules, the blab output
doesn't have those core rules defined, so blab can't parse the fule:

$> echo "a = DIGIT" | kgt -l abnf -e blab | blab
Could not find /usr/share/blab/DIGIT.blab needed for 'DIGIT'

Promote the rrd node tree to serve as a general IR for all dialects

The AST is inconvenient; it has nesting of a forced hierarchy of particular types. In retrospect I think I'd prefer to construct rrd nodes directly. And now the tnode tree has positional information split off, I think these rrd nodes should serve well as an abstract representation of grammars for both transformation and for rendering to all outputs. In turn that makes the AST nodes redundant, and I think those can go.

So:

Move the rrd nodes out for general use;
Switch all outputs to the rrd nodes;
Have all outputs share the tree rewriting;
Construct the rrd nodes directly from parser actions;
Remove AST;

And perhaps:
6. Rename rrd nodes to AST
7. Rename tnode to rrd?

Feature Request: Configurable production rule separator

If the production rule separator was configurable or optional, it would make it easier to use nonstandard grammars from various sources.

Text output for Model 33 ASR teletypes

When writing RFCs, we are limited to 69 columns (72 columns in an emergency) for plaintext output (SVG for HTML and PDF are unrestricted).

I can easily squash 4 columns by removing the indentation for the diagram:

cri-reference:
    ╭──────────────────────────────────────>───────────────────────────────────────╮
    │                                                                              │
    │                               ╭─────────────────────>─────────────────────╮  │
    │                               │                                           │  │
    │                               │          ╭──────────────>──────────────╮  │  │
    │                               │          │                             │  │  │
    │                               │          │           ╭──────>───────╮  │  │  │
    │                               │          │           │              │  │  │  │
│├──╯──╮── scheme ── authority ──╭──╯── path ──╯── query ──╯── fragment ──╰──╰──╰──╰──┤│
       │                         │
       ╰──────── discard ────────╯

The ideal way to handle this would be to be able to give a column limit to rrtext/rrutf8 and let kgt figure it out...

A simple space-saving flag (e.g., remove spaces around scheme, authority..., making │├── and ──┤│ vertical) would already help.

Feature Request: Configurable "equality" symbol in production rule

Many grammars use different equality symbols to separate the left and right sides of a production rule:
symbol ::= replacement
symbol = replacement
symbol → replacement

If this symbol was configurable, it would make it easier to use these grammars with the tool.

Implement # for ABNF

RFC7230 defines the extension # which provides lists:

     1#element => element *( OWS "," OWS element )
   and:
     #element => [ 1#element ]
   and for n >= 1 and m > 1:
     <n>#<m>element => element <n-1>*<m-1>( OWS "," OWS element )
...

     #element => [ ( "," / element ) *( OWS "," [ OWS element ] ) ]
     1#element => *( "," OWS ) element *( OWS "," [ OWS element ] )

https://tools.ietf.org/html/rfc7230#section-7

ABNF: comments

ABNF allows comments using ; as a delimiter to signal a comment to the end of the line.

This seems to be supported inline in a rule; e.g., this works:

foorule = bar  ; this is a comment
        / baz

but if the comment is on the last line of a rule, it will fail; e.g., either of these forms:

foorule = bar ; this is a comment

foorule = bar ; this is a comment
        / baz ; more commenting

Cannot generate railroad diagrams for ABNF files from the examples

I tried to generate railroad diagram for the TOML file format's ABNF grammar and it failed silently:

$ kgt -l abnf -e rrtext < toml.abnf

There are 5 ABNF example files that ship with kgt:

$ ls -1 examples/*.abnf
examples/abnf.abnf
examples/datetime.abnf
examples/irc.abnf
examples/postal.abnf
examples/utf8.abnf

kgt is able to generate railroad diagrams only for 2 of them. The following 3 ABNF files do not product any output:

$ kgt -l abnf -e rrtext < examples/abnf.abnf
$ kgt -l abnf -e rrtext < examples/irc.abnf
$ kgt -l abnf -e rrtext < examples/postal.abnf

What am I doing wrong? 🤔

Blab output - grouping is not perserved

Given this example:

echo -e 'b = a " "\n\na = ("1" / "2") "3"' | kgt -l abnf -e blab | blab -n 10

The output should be:

13 23 23 23 13 23 23 13 23 23

But instead, we get:

1 23 23 23 1 23 23 1 23 23

Because the generated blab output is:

b = a  " "

a = "1" | "2"  "3"

Rather than (something like):

b = a  " "

a = ("1" | "2")  "3"

ABNF: prose values

ABNF allows a prose-val to be used:

         prose-val      =  "<" *(%x20-3D / %x3F-7E) ">"
                                ; bracketed string of SP and VCHAR
                                ;  without angles
                                ; prose description, to be used as
                                ;  last resort

This is commonly used in IETF RFCs (e.g., for references to rules defined in other specifications). However, it doesn't seem to be supported.

Nearly all ebnf files failed to work?

in the example directory, I tried to use the ebnf format.
They are so inconsistent, some times need a ',', sometimes not necessary.
Only the json.ebnf can pass the generator.
the format: ALPHA not work, need to use 0ALPHA
1*4DIGIT not work at all, may be it means DIGIT{1,4} in perl re style.
=/ also not work.
The generation in rrtext:
{} and [] make no difference, only U and upsidedown U, I think there are some common bugs
that trigger all the format failed also. Sorry for so many complain.

Feature request: add language from the "Railroad-diagram Generator"

Add support for the language from the Railroad-diagram Generator as input.

Example:

Diagram(
  Optional('+', 'skip'),
  Choice(0,
    NonTerminal('name-start char'),
    NonTerminal('escape')),
  ZeroOrMore(
    Choice(0,
      NonTerminal('name char'),
      NonTerminal('escape'))))

Online:
https://tabatkins.github.io/railroad-diagrams/generator.html

Format documentation:
https://github.com/tabatkins/railroad-diagrams#components

How can I compile the kgt adn libfsm under linux?

I failed to compiler the kgt and libfsm, both needed a pmake program.
My archlinux system only have gnu make, bmake, no pmake,
Can I compile the C source code manually without the makefile,
I cannot find the lexer .lx and parser .sid files program,
What parser and lexer generator are you used for this program. Thanks for your attention.

A possible test file

In the repository

https://github.com/katahiromz/EbnfParser
ISO EBNF notation parser in C++
( License : CC0 1.0 Universal )

there is a ISO EBNF conforming file :

c99-grammar.txt

With the following statement :

[s@localhost kgt]$ Installed/bin/kgt -l iso-ebnf -e dot < c99-grammar.txt > c99-grammar.dot
[s@localhost kgt]$

[s@localhost kgt]$ uname -a
Linux localhost.localdomain 4.18.11-200.fc28.x86_64 #1 SMP Sun Sep 30 15:31:40 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[s@localhost kgt]$

Fedora 28

it produced a viewable ( without error message ) a .dot file .

Very beautiful and pleasing ...

Bugs found in rrtext output.

Hi:
A bug is found:
when using rrtext with the following example:
p = [a] , [b] ;

kgt -l ebnf -e rrtext < bug.ebnf >bug.txt

bug.ebnf.txt
patch.txt

There are trailing spaces at the end of output:

p:
>-------v >-------v <- trailing spaces
| | | | <- trailing spaces
||--^-- a -->--^-- b -->--||

if the production rule output is shorter than the console width, it's not
obvious, with a long production rule, like:

block = [const-declaration] ,
[var-declaration] ,
[proc-declaration] ,
compund-statement
;

break of lines occurred.

So, some fix required. I made a simple patch for temporary fix.
Just right trim the output string:

#include <ctype.h>

char *rtrim(char *s) {
int n = strlen(s);
while(--n>=0 && ( iscntrl(s[n]) || isspace(s[n]) || isblank(s[n])))
;
s[++n] = 0;
return s;
}

or a patch file attached for the src/rrtext/output.c

Thanks.

ABNF: support case sensitive strings

%s"abc" and %i"abc" mean case sensitive and insensitive respectively.

I think these would best be provided either as a distinct AST node, or as a boolean on the existing node. My main requirement is to make RRD transformations with adjacent strings (factoring out common prefixes, for example) without too much hassle.

Then how to display them can be deferred to the tnode tree. Perhaps the RRD tree would have one node type carrying a boolean, and the tnode tree would have two node types, constructed as an alt.

https://tools.ietf.org/html/rfc7405

katef / kgt Goto Github PK

kgt's People

Contributors

Stargazers

Watchers

Forkers

kgt's Issues

Recommend Projects

Recommend Topics

Recommend Org