Hello Mike,

I was wandering if there is any way how to export Parse Tree with that "interactive highlights" (when hovering above categories nodes) as now if I am correct it "only" (very impressive and cool regardless) as static SVG. Is there at least any way how to hack it in any way or form into at least half decent format (I assume combination of js and html is in place right now)

Thanks for any help!

I'm trying to use antlr4.generation.outputDir to prevent the plugin from making a .antlr directory relative to my antlr files but changing it doesn't seem to work. I've restarted VSCode after updating the config and I'm still getting the .antlr directory.

For what it is worth I think creating a directory next to the source code isn't a great default either, but I'm happy to use the setting to move it. I just can't figure out how to get it working.

When antlr4.generation.outputDir is set to ".antlr" and antlr4.generation.mode is set to "external", an error pops up:

Error while transfering interpreter data: Error: Source and destination must not be the same.

https://github.com/mike-lischke/vscode-antlr4/blob/master/src/extension.ts#L382-L396

https://github.com/jprichardson/node-fs-extra/blob/fab376ec2bd4d4420806644c1b37a7ac487d17ec/lib/copy-sync/copy-sync.js#L29-L30

ctdLexer.g4

lexer grammar ctdLexer;
DOT: '.';

ctdParser.g4

parser grammar ctdParser;
main: DOT;

it always says:

warning(125): c:\Users\*******\test\grammar\ctdParser.g4:2:6: implicit definition of token DOT in parser

{
    "version": "2.0.0",
    "configurations": [
        {
            "name": "test",
            "type": "antlr-debug",
            "request": "launch",
            "input": "input.txt",
            "grammar": "grammar/ctdParser.g4",
            "startRule": "main",
            "printParseTree": false,
            "visualParseTree": false,
            "windows": {
                "trace": false
            }
        }
    ]
}

debugging throws this error:

Error while launching debug session: TypeError: Cannot set property 'inputStream' of undefined

Am i missing something?

When setting antlr4.format.columnLimit to 0, the format becomes weird:

comparing to what I expect:

My configuration is like below:

{
    "antlr4.format.alignColons": "hanging",
    "antlr4.format.alignSemicolons": "ownLine",
    "antlr4.format.allowShortRulesOnASingleLine": false,
    //"antlr4.format.columnLimit": 0,
    "antlr4.format.columnLimit": 10000,
    "antlr4.format.useTab": false,
    "antlr4.generation.mode": "external",
    "antlr4.generation.outputDir": "/.antlr",
}

I really appreciate this great project but there is a need for a quickstart: ok, I installed the extension, I see the syntax highlight but where is the other stuff? Do I need ANTLR preinstalled? If so, how can I reference the ANTLR installation to this extension?

With just a few steps in a numbered list and prerequisites the experience would be greater and the audience can grow even more! I think this saves neuron energy consumption where there is no need to think.

Thank you.

I really liked this plugin but I really miss the parseTree view I had back in IntelliJ and Eclipse.
Basically given a grammar and example source input it display the AST graphically.
ideally, the example source input could be a file or a text area + with live update over the AST graph view.

If this feature is already in please let me know who to fire it up.

Many Thanks
Felipe

Hi,
I'm having an issue using the nuget Antlr4 version 4.6.5-beta002 with the extension. I think it's due to a version mismatch. The extension seems to be generating code for antlr 4.7.2.

I just updated to 2.0.0. But I see no more setting entry to configure the output folder for generated parser.

Did I miss something? I only see below config entries:

  // Max number of labels displayed on a transition in an ATN graph
  "antlr4.atn.maxLabelCount": 3,

  // Default export target folder for ATN SVG files
  "antlr4.atn.saveDir": "",

  // Default export target folder for call graphs
  "antlr4.call-graph.saveDir": "",

  // List of custom CSS URIs for diagram SVG files
  "antlr4.customcss": [],

  // Debugging related settings
  "antlr4.debug": {},

  // Settings related to parser generation
  "antlr4.format": {
    "alignTrailingComments": false,
    "allowShortBlocksOnASingleLine": true,
    "breakBeforeBraces": false,
    "columnLimit": 100,
    "continuationIndentWidth": 4,
    "indentWidth": 4,
    "keepEmptyLinesAtTheStartOfBlocks": false,
    "maxEmptyLinesToKeep": 1,
    "reflowComments": true,
    "spaceBeforeAssignmentOperators": true,
    "tabWidth": 4,
    "useTab": true,
    "alignColons": "none",
    "singleLineOverrulesHangingColon": true,
    "allowShortRulesOnASingleLine": true,
    "alignSemicolons": "none",
    "breakBeforeParens": false,
    "ruleInternalsOnSingleLine": false,
    "minEmptyLines": 0,
    "groupedAlignments": true,
    "alignFirstTokens": false,
    "alignLexerCommands": false,
    "alignActions": false,
    "alignLabels": true,
    "alignTrailers": false
  },

  // Settings related to parser generation
  "antlr4.generation": {
    "mode": "internal",
    "language": "Java",
    "listeners": true,
    "visitors": false
  },

  // Enable/disable the references code lens
  "antlr4.referencesCodeLens.enabled": false,

  // Default export target folder for railroad diagrams
  "antlr4.rrd.saveDir": ""

Hello Mike,

I love your extension and its absolutely genius, however I already struggled a bit with making it work even on a dummy test project and I cannot get a graph visualization or cannot find the way how to invoke / trigger it in VS.

Took mi already few hours to translate "open via context menu of IDE" to 3 dots in the end of open files tab, there I found the Railroad diagram, but not the ParseTree Graph.

Also I am getting print tree correctly in Debug Console so probably only the case of finding how to show it in VS Code?

Config should be alright:
{
"name": "antlr4",
"type": "antlr-debug",
"request": "launch",
//"input": "${workspaceFolder}/${command:AskForTestInput}",
"input": "input.txt",
"grammar": "Expression.g4",
"startRule": "startWithEOF",
"visualParseTree": true,
"printParseTree": true
}

Best regards,
Simon

When dealing with separate lexer and parser grammar, tokens added to lexer file after tokenVocab in parser file is specified will cause "Unknown token reference" error, even after regeneration (clean .antlr folder and save lexer/parser files again, or manually call antlr4 from terminal, anyway).

I was developing ANTLR4 Grammar projects, however, came up with the question of whether it was possible to create a Language Server for an ANTLR4-based language that I've created. If so, what is the procedure to do that?
I'm still very new to ANTLR4. Your help is greatly appreciated!

happens after commit c185175#diff-9e104f81fe91f5fa268b1d7193109c23

Plugin looks great.

How do I

1. Set the output path? The ".antlr" directory doesn't work well with my webpack setup, I need to place the sources under "../src"

2. How do I configure it to generate Visitor and Listener interfaces?

Thanks!

Having a textbox in where you can type and see the parsing results while typing could be very helpful.
Also the other way around, in where the input text gets parsed while the grammar is being updated.

Not sure if it can be done, but it could be helpful.

See attached images

Hello,

I'm using the extension with 2 computers using Windows 10 Pro 64 bits. On my Surface pro, it's working fine generating the sources. But on my desktop computer, I have ANTRL in the status bar with the progress indicator but it doesn't generate the sources.

In the output console, I have this error:
[2018-02-17 08:58:19.214] [renderer1] [error] spawn java ENOENT : Error: spawn java ENOENT
at exports._errnoException (util.js:1050:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
at onErrorNT (internal/child_process.js:367:16)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickCallback (internal/process/next_tick.js:104:9)

I'm using Java 8 on both computer.

Regards,
Stef

I have javascript actions that look like this:

element_modification returns [v]
   : name { $v = { name: $name.text }; } (modification { $v.modification = $modification.v; } )? string_comment { $v.comment = $string_comment.v; }
   ;

The nested object literals with curly braces cause the above line to be highlighted incorrectly.

If I modify antlr.json to make "action" recursive it fixes it:

"action": {
  "name": "entity.other.block.antlr",
  "begin": "\\{",
  "end": "\\}",
  "patterns" : [
    { "include" : "#action" }
  ]
},

Not sure if this is the best thing to do though? I've never really messed with these textmate files before.

The visual example (sry the quality is suck):

Well, I can't tell whether this is a bug of the extension or vsc itself. Anyway, please try to reproduce this to figure out.

After I right click a rule and select "Show ATN graph for rule", I only got this:

I have the issue mentioned in the title in the current version in the VS Marketplace (2.0.4).

MCVE:

grammar:

grammar TreeVisualizationFail;

LETTER : [a-zA-Z];
DIGIT : [0-9];

 
startRule : digits letters EOF;

letters : LETTER+;

digits : DIGIT*;

input:

foo

Note that for the given input the mandatory rule digits in startRule matches no tokens, i.e. an empty string.

After making digits optional in startRule and forcing digits to match at least one digit, the parse tree visualization gets generated.

Originally, I encountered this problem with a grammar used to parse VBA.

When debugging, the graph nodes cover lines and the tokens (see image).

It would be good to be able to have settings in the configuration to:

1. increase the separation
2. choose straight lines or the current ones
3. colors for lines, tokens and nodes
4. collapse rules

This is a weird issue, when using semantic predicates, the generated grammar behaves incorrectly in debugging mode, a good example from the ANTLR4 book:

grammar Data;

file: group+;
group: INT sequence[$INT.int];

sequence[int n]
	locals[int i = 1;]: ( {$i<=$n}? INT {$i++;})*;

INT: [0-9]+;
WS: [ \t\n\r]+ -> skip;

Running against this sample data:

1 23 3 6 8 4

Against antlr4 tools (4.7.1) returns correctly:

But the same parse tree is incorrect in VSCode ANTLR4:

I think it has something to do with ANTLR binaries, because if I try the same in latest IntelliJ ANTLR4 plugin (version 1.8.4) I see the same invalid tree:

For example, in the following railroad diagram:

there should be some way to expand the LineContinuation node to its full definition, in-place:

One issue: how should the expand be triggered? If this was the only possible interaction, then I would suggest clicking on the node should expand (what about collapse). However, perhaps clicking should be reserved for navigation within the railroad diagram view (#25); this would require some kind of additional UI.

antlr/antlr4#1995

The plugin should probably only support generating listeners for now, since the visitor implementation is completely bogus.

On Windows 10 I selected a rule from a parser g4 file, and right click to fire "Show ATN ...", but got "No ATN data found (code generation must run at least once in internal or external mode)"

I do understand this message, but how do I run the code gen?

I write a vscode extension.

and now I need run a button click function.

but

Is there any solution?

When I set mode to 'external' to generate JavaScript output I have to manually copy the .interp files into the .antlr folder to get the debugger to run. Without the the files in the .antlr folder I get the following error message when trying to run the debugger.

I'm using version 2.0.2.

When file contains final newline, formatting will increase the number of final newlines to min(antlr4.format.minEmptyLines, antlr4.format.maxEmptyLinesToKeep) + 1. It won't happen if there's no final newline in the file.
Maybe an upstream bug?

I haven't looked closely at vscode-antlr4, but I am using it. Thanks Mike.

I noticed generated files getting added to a .antlr directory under where my .g4 files are located. I'm guessing this came from vscode-antlr4, right?

I'm not saying that is a bug, in fact I was thinking about doing something similar in antlr4ts-cli, though I considered putting it in a temp directory outside the source tree. Is there some reason you chose to put it into the source directory? Any related thoughts?

I'm also interested in the ".interp" file format generated in this directory. Can you point me at anything about this format?

I’ve created a .g file which contains grammar rules but I don’t know how to parse input file and display graphs.
I would be really great full if you could you help me.

1. Put the following in a .g4 file
2. Open in vscode
3. format doc (SHIFT+ALT+F, or save if auto format is on)
   !) Everything from the antlr-format line and on is deleted

grammar mongo;

// $antlr-format off

@lexer::members {
	private isExternalIdentifierText(text) {
		return text === 'db';
	}
}

mongoCommands: EOF;

Allow navigating to a child node in the railroad diagram view, from the railroad diagram view.

I think the natural interaction choice would be to click on the node.

Perhaps this could be implemented by passing some sort of dummy URL as the second parameter to the JS library used to generate the diagrams, and registering a global handler for this dummy URL to navigate to the appropriate node.

Trying to learn ANTLR and enjoying this extension greatly for debugging grammar. I'm getting to the point I'm wanting to start integrating the grammar into a project and while I see settings for configuring the language output, from the sparse documentation it is unclear to me how to configure the extension to output additional grammars (where should the setting file be placed in the folder & how . Is this possible or do I need to run the grammar through the command line?

It would be nice if the builds were in the same project so that I can iterate between the without moving back & forth.

When debugging an input file, the parse tree is written to files that all have the same name "Parse Tree". This makes it very difficult to keep track of which file goes with which run. As an example, if I want to compare the results of parsing 2 separate input files, it quickly becomes unmanageable and significantly interrupts the typical workflow.

Instead of displaying the parse tree in a separate file, is it possible to:

1. Create a split pane within the input file window itself to display the parse tree for that file? or
2. Rename the parse tree file to match that of the input file (e.g. "Parse Tree - My Input File")?

None of the graphs are rendering (parse tree, rule, call graph). Just a grey box is shown.

Grammar (datadef.g4)

grammar datadef;

file: tabledef*;
tabledef: ID '{' ('types' '{' typedef (',' typedef)* '}')? memdef (',' memdef)* '}' ';';
typedef: ID ':' typeref;
memdef: ID ':' typeref;
typeref: (ID | PRIMITIVE)+;

PRIMITIVE: 'str' | 'num';
ID: [a-zA-Z_][a-zA-Z0-9_-]*;
BLOCK_COMMENT: '/*' .*? '*/' -> skip;
LINE_COMMENT: '//' ~[\r\n]* -> skip;
WHITESPACE: [ \t\f\r\n] -> channel(HIDDEN);
INVALID_INPUT: [\u0001-\u0008] | '\u000B' | '\u000C' | [\u000E-\u001F];

Test input (input.def)

MyTable {
    types {
        type1 : num str num,   // comment 1
        type2 : num type1 str
    }
    mem1: num str type1,
    mem2: num,
    mem3: type1 type1 type2
};

// comment 2

Debug configuration

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "antlr4-datadef",
            "type": "antlr-debug",
            "request": "launch",
            "input": "input.def",
            "grammar": "datadef.g4",
            "startRule": "file",
            "printParseTree": true,
            "visualParseTree": true
        }
    ]
}

Hi,

up front let me say thank you very much for this fantastic extension!

I have configured the launch task according to documentation, and it prints correctly to the debug console, visual Parse Tree does not work though:

    "version": "2.0.0",
    "configurations": [        
        {
            "name": "antlr4-flow",
            "type": "antlr-debug",
            "request": "launch",
            //"input": "${workspaceFolder}/${command:AskForTestInput}",
            "input": "test.flow",
            "grammar": "flow.g4",
            "startRule": "file",
            "printParseTree": true,
            "visualParseTree": true
        }
    ]

If I set "visualParseTree" to false the debugger will start, print results to debug console and exit.

If it is set to true the debugger will start, print results to the debug console and then apparently get stuck (the toolbar with pause, restart, stop buttons is displayed but no visual parse tree appears).

Best regards,
Robert.

When using 2 separate files for Parser and Lexer, after the Lexer rule is changed, the IDE picks up on the change but the debugger doesn't. It displays the diagram with the old Lexer rules. Deleting the generated .antlr folder makes no difference, the only thing that helps is re-starting VS Code.

VS Code 1.26.1

> java -version
java version "10.0.2" 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

antlr-4.7.1-complete

Windows 8.1

Hi,
I'm getting some troubles when I try to launch a debug session.
Launch.json:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "antlr-debug",
            "request": "launch",
            "name": "Run Parser2",
            "input": "Samples/TestAssignment.ls",
            "visualParseTree": true
        },
        {
            "name": "Run Parser",
            "type": "antlr-debug",
            "request": "launch",
            "grammar": "AntlrFanuc.g4",
            //"startRule": "0",
            "startRule": "moduleDeclaration",
            //"input": "${workspaceFolder}/${command:AskForTestInput}",
            "input": "Samples/TestAssignment.ls",
            "stopOnEntry"  : true,

            "printParseTree": true,
            "visualParseTree": true,
            "trace": true
        }
    ]
}

Look's like the grammar is not compiling.
I both cases no start rule is found.
Am I missing something?

Hi 👋

That would be great to disable autofocus on the output view when the extension displays a message. It is very frustrating to change constantly between tabs. Errors are already displayed in the error view, so I don't think it's useful to keep the current behaviour.

It seems to be possible in the VS Code API by saying preserveFocus: true

https://github.com/Microsoft/vscode/blob/e48a3284f426af702223ff9aae4f5662dd29792d/src/vs/vscode.d.ts#L3897-L3902

Would be awesome. Thanks!

After installing the September, 2018 release of VSCode (version 1.28), the antlr4 debugging fails to run with the following error message: "Debug adapter process has terminated unexpectedly (read error)".

Hi! Thanks for nice extension. But I have some issues with it. The most important for me now is not working debugger. It worked earlier, but I edited launch configuration and VS showed me the error: "Debug adapter process has terminated unexpectedly". I returned my launch configuration back, but it didn't help. My current launch configuration is:

        {
            "name": "ANTLR",
            "type": "antlr-debug",
            "request": "launch",
            //"input": "${workspaceFolder}/${command:AskForTestInput}",
            "input": "tests/input.qs",
            "grammar": "QaterParser.g4",
            "startRule": "program",
            "printParseTree": true, // I removed this line when I got the error in first time
            "visualParseTree": true
        },

My workspace config is the following:

    "antlr4.generation": {
        "mode": "internal",
        "outputDir": "generated",
        "language": "Cpp", // I know this is not used when mode is 'internal'
        "listeners": false,
        "visitors": true
    },

VS Code version: 1.21.0
Architecture: x64
Extension version: 1.3.0

I've opened the VB.NET grammar (from here -- link at the bottom of the page). The ANTLR Exception output pane shows errors:

warning(184): d:\Zev\Projects\vbnet-grammer\vb.g4:2274:0: One of the token XMLNameStartCharacter values unreachable. _ is always overlapped by token XMLNameCharacter
warning(184): d:\Zev\Projects\vbnet-grammer\vb.g4:2274:0: One of the token XMLNameStartCharacter values unreachable. : is always overlapped by token XMLNameCharacter
warning(184): d:\Zev\Projects\vbnet-grammer\vb.g4:2496:0: One of the token XMLNamespaceNameCharacter values unreachable. _ is always overlapped by token XMLNameCharacter
warning(184): d:\Zev\Projects\vbnet-grammer\vb.g4:2496:0: One of the token XMLNamespaceNameCharacter values unreachable. _ is always overlapped by token XMLNameStartCharacter
error(119): d:\Zev\Projects\vbnet-grammer\vb.g4::: The following sets of rules are mutually left-recursive [CCExpression, CCOperatorExpression] and [Expression, IsExpression, MemberAccessBase, DictionaryAccessExpression, InvocationExpression, IndexExpression, AdditionOperatorExpression, SubtractionOperatorExpression, MultiplicationOperatorExpression, FPDivisionOperatorExpression, IntegerDivisionOperatorExpression, ModuloOperatorExpression, ExponentOperatorExpression, RelationalOperatorExpression, LikeOperatorExpression, ConcatenationOperatorExpression, ShortCircuitLogicalOperatorExpression, LogicalOperatorExpression, ShiftOperatorExpression, XMLMemberAccessExpression, TypeExpression, MemberAccessExpression, OperatorExpression, ArithmeticOperatorExpression, DivisionOperatorExpression]

However, I don't see any of these errors in the Problems view, or inline in the editor.

I prefer to view the parse tree vertically, but there doesn't seem to be a setting that specifies the default orientation. Consequently I have to manually switch to vertical for every run.

Similarly, it would be useful to specify a default "Tree/Cluster" setting.

It's possible that some rule names may conflict with generated code in target language, e.g. operator in C++, while they are valid identifier names in Java.

Hi,

I'm getting the following error

[Internal Error] Unhandled error message (code 50, message: syntax error: mismatched character '<EOF>' expecting '"'

on this grammar

program 
    : (description NL+)? stmts* EOF
    ;

description
    : 'desc' WS+ string NL*
    ;

stmts
    : stmt NL*
    | stmts NL+ stmt NL+
    ;

stmt
    : assignment
    | return
    | expr
    ;

expr
    : number
    | identifier
    | fapply
    | addexpr
    ;

addexpr
    : mulexpr
    | addexpr WS* '+' WS* mulexpr
    | addexpr WS* '-' WS* mulexpr
    ;

mulexpr
    : unary
    | mulexpr WS* '*' WS* unary
    | mulexpr WS* '/' WS* unary
    ;

fapply
    : identifier '(' ( ')' | arglist ')' )
    ;

assignment
    : identifier WS* '=' WS* expr
    ;
return
    : 'return' WS* expr
    ;

arglist
    : expr
    | arglist WS* ',' WS* expr
    ;

unary
    : number
    | identifier
    ;

primary
    : number
    | string
    | identifier
    ;

string
    : '"' [a-zA-Z-_ ()']+ '"'
    ;

number
    : [+-]?([1-9][0-9]+|[0-9])
    ;

identifier
    : [a-zA-Z_][a-zA-Z0-9_]+
    ;

NL  : '\r'? '\n' -> skip
    ;

WS  : [ \t\r\u000C]+ -> skip
    ;

I ported this grammar from an instaparse source grammar file, so it may be broken in a number of ways.
Let me know if you need help/more info.

Regards,
Michelangelo

When there are two unrelated grammars in the same directory the same rules name in both files causes a Duplicate symbol error. Both grammars are totally valid and the Antlr tools is fine with them.

It would be nice in the plugin didn't complain.

Currently, with input and grammar entries in launch.json you cannot use ${} variables, for example, the following sample:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "antlr4",
      "input": "${workspaceFolder}/${command:AskForTestInput}",
      "type": "antlr-debug",
      "request": "launch",
      "grammar": "Data.g4",
      "startRule": "${command:AskForTestInput}",
      "printParseTree": true,
      "visualParseTree": true
    }
  ]
}

will fail with:

Cannot read test input file:
[some directory]/${workspaceFolder}/${command:AskForTestInput}

The same is for grammar while startRule works perfectly, for example, this launch configuration will work:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "antlr4",
      "input": "test.data",
      "type": "antlr-debug",
      "request": "launch",
      "grammar": "Data.g4",
      "startRule": "${command:AskForTestInput}",
      "printParseTree": true,
      "visualParseTree": true
    }
  ]
}

I tried with ${file} and similar, but it looks like grammar and input are not preprocessing macros.

Tested in version v2.0.4