Code Monkey home page Code Monkey logo

a-lpg / lpg2 Goto Github PK

View Code? Open in Web Editor NEW
5.0 1.0 4.0 3.79 MB

The LALR parser generator (LPG) is a tool for developing scanners and parsers. Supports multi-language . Input is specified by BNF rules. LPG supports backtracking (to resolve ambiguity), automatic AST generation and grammar inheritance.

License: Eclipse Public License 2.0

GAP 27.11% C++ 72.73% C 0.03% CMake 0.12%
parser-generator grammar dfa lalr language-recognition grammar-parser lalr-parser-generator scannerless yacc-lex jikespg

lpg2's Introduction

LPG2

The LALR parser generator (LPG also known as JikePG) is a tool for developing scanners and parsers . Input is specified by BNF rules. LPG supports backtracking (to resolve ambiguity), automatic AST generation and grammar inheritance.

Automatic AST generation Supports multi-language.

- Java
- JavaScript
- TypeScript
- Python
- Dart
- Go
- C++
- C#

LALR table Supports multi-language.

- Java
- JavaScript
- TypeScript
- Python
- Dart
- Go
- C++/C
- C#

Getting Started with LPG

Getting Started with LPG

LPG2's history

This is a fork of LPG which create by: Philiphe Charles, Robert M. Fuhrer , Gerry Fisher. But It didn't update since 2013-04-25. We are the user of this tool. We want to update this tool , so we fork it and update it here.the original fork only support java runtime . We brought the lsp server , vscode extend ,other language runtime for LPG2 .

Example Uses

  • Eclipse IDE Meta-tooling Platform IMP radically simplifies and speeds up the IDE development process in Eclipse, for both language with existing front-ends as well as languages generated using compiler and interpreter generation frameworks.
  • jikes Jikes is a java compiler .
  • JCDT The core part of JCIDE v3.
  • LPG2 is itself implemented in LPG2.
  • LPG-language-server LPG-language-server for LPG
  • eclipse.jdt.core The core part of Eclipse's Java development tools

lpg2's People

Contributors

kriegaex avatar kuafuwang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

lpg2's Issues

Get invalid generated code by having two conflicting function definitions when option -> glr is true:

It seems that you have missed the fixes for this #1 (comment) see here

if (option -> glr)
, without that fix (on several *Action.cpp it seems that we'll get invalid generated code by having two conflicting function definitions when option -> glr is true:

if (option -> glr)
    {
        b.Put(indentation); b.Put("    Ast* nextAst = nullptr;\n");
        b.Put(indentation); b.Put("    IAst* getNextAst() { return nextAst; }\n"); ///<<<< first definition
        b.Put(indentation); b.Put("    void setNextAst(IAst* n) { nextAst = n; }\n");
        b.Put(indentation); b.Put("    void resetNextAst() { nextAst = nullptr; }\n");
    }
    else b.Put(indentation); b.Put("    IAst* getNextAst() { return nullptr; }\n"); ///<<<< second conflicting definition

Originally posted by @mingodad in #1 (comment)

Relationship with https://github.com/daveshields/jikespg ?

I'm trying to learn a bit more about the world of parser generators. I know Eclipse JDT uses jikespg (an old 1.3 version) to generate the parser ( https://www.eclipse.org/jdt/core/howto/generate%20parser/generateParser.html ), I also found https://github.com/daveshields/jikespg ; and then this repo says "The LALR parser generator (LPG also known as JikePG)".
So do you think that the parser generator here could be usable an alternative to jikespg in Eclipse, using the same grammar file ( https://github.com/eclipse-jdt/eclipse.jdt.core/blob/master/org.eclipse.jdt.core.compiler.batch/grammar/java.g ) and lead to same -or better- result?

Segfault on linux64

Trying to build this project on Ububtu 18.04 I needed to make several changes to be able to build it, also several chnges have been made to remove several compiler warnings with `-Wall -Wextra".
When ti first build it was segfaulting here

act = nt_action(stack[state_stack_top], lhs[act]);
I've added a test before doing the call and now it's not segfaulting but the output doesn't seem ok.

I'm attaching the output of git diff -u with my changes, just in case it can help.

lpg2-dad.diff.zip

Amount of conflicts differ from bison?

Hello @kuafuwang,

I have a quick question and I hope you don't mind if I ask here.

I've noticed that LPG2 reports fewer conflicts with LA=1 than, for example, bison set to LALR(1).

I'm wondering, do you know if there are other techniques that LPG2 uses to remove conflicts (by default), and is there maybe a way to turn them all off?

I'm trying to get LPG2 to report the same amount of conflicts that bison does for LALR(1), but that's never the case. Do you know what could be going on?

Here's, for example, what bison reports for a grammar:

Screenshot 2023-12-11 at 01 51 37

And here's what LPG2 reports:

Screenshot 2023-12-11 at 01 51 29 Screenshot 2023-12-11 at 01 51 09

Notice how LPG2 reports fewer than 100 conflicts, but bison reports almost 500 conflicts.

Grammar railroad diagram

Based on the code from void Grammar::DisplayInput(void) I built so far this initial EBNF grammar (after some small manual changes) that can be used on https://www.bottlecaps.de/rr/ui to get a railroad diagram (https://en.wikipedia.org/wiki/Syntax_diagram).

I think that it's a useful option to have on LPG2.

Copy and paste the EBNF shown bellow on https://www.bottlecaps.de/rr/ui in the TAB Edit Grammar then switch to the TAB View Diagram.

//EBNF to be viewed with https://www.bottlecaps.de/rr/ui

JikesPG_INPUT ::= Grammar
 Grammar ::= //$Empty
 Grammar ::= Grammar include_segment END_KEY_opt
 Grammar ::= Grammar notice_segment END_KEY_opt
 Grammar ::= Grammar define_segment END_KEY_opt
 Grammar ::= Grammar terminals_segment END_KEY_opt
 Grammar ::= Grammar export_segment END_KEY_opt
 Grammar ::= Grammar import_segment END_KEY_opt
 Grammar ::= Grammar softkeywords_segment END_KEY_opt
 Grammar ::= Grammar eof_segment END_KEY_opt
 Grammar ::= Grammar eol_segment END_KEY_opt
 Grammar ::= Grammar error_segment END_KEY_opt
 Grammar ::= Grammar recover_segment END_KEY_opt
 Grammar ::= Grammar identifier_segment END_KEY_opt
 Grammar ::= Grammar start_segment END_KEY_opt
 Grammar ::= Grammar alias_segment END_KEY_opt
 Grammar ::= Grammar names_segment END_KEY_opt
 Grammar ::= Grammar headers_segment END_KEY_opt
 Grammar ::= Grammar ast_segment END_KEY_opt
 Grammar ::= Grammar globals_segment END_KEY_opt
 Grammar ::= Grammar trailers_segment END_KEY_opt
 Grammar ::= Grammar rules_segment END_KEY_opt
 Grammar ::= Grammar types_segment END_KEY_opt
 Grammar ::= Grammar dps_segment END_KEY_opt
 include_segment ::= 'INCLUDE_KEY'
 include_segment ::= 'INCLUDE_KEY' 'SYMBOL'
 notice_segment ::= 'NOTICE_KEY'
 notice_segment ::= notice_segment action_segment
 define_segment ::= 'DEFINE_KEY'
 define_segment ::= define_segment macro_name_symbol macro_segment
 macro_name_symbol ::= 'MACRO_NAME'
 macro_name_symbol ::= 'SYMBOL'
 macro_segment ::= 'BLOCK'
 terminals_segment ::= 'TERMINALS_KEY'
 terminals_segment ::= terminals_segment terminal_symbol
 terminals_segment ::= terminals_segment terminal_symbol produces name
 export_segment ::= 'EXPORT_KEY'
 export_segment ::= export_segment terminal_symbol
 import_segment ::= 'IMPORT_KEY'
 import_segment ::= 'IMPORT_KEY' 'SYMBOL' drop_command_list
 drop_command ::= drop_symbols
 drop_command ::= drop_rules
 drop_command ::= 'DROPACTIONS_KEY'
 drop_symbols ::= 'DROPSYMBOLS_KEY'
 drop_symbols ::= drop_symbols 'SYMBOL'
 drop_rules ::= 'DROPRULES_KEY'
 drop_rules ::= drop_rules drop_rule
 drop_rule ::= 'SYMBOL' produces rhs
 drop_rule ::= 'SYMBOL' 'MACRO_NAME' produces rhs
 drop_rule ::= drop_rule '|' rhs
 drop_command_list ::= //$Empty
 drop_command_list ::= drop_command_list drop_command
 softkeywords_segment ::= 'SOFTKEYWORDS_KEY'
 softkeywords_segment ::= softkeywords_segment terminal_symbol
 softkeywords_segment ::= softkeywords_segment terminal_symbol produces name
 error_segment ::= 'ERROR_KEY'
 error_segment ::= 'ERROR_KEY' terminal_symbol
 recover_segment ::= 'RECOVER_KEY'
 recover_segment ::= recover_segment terminal_symbol
 identifier_segment ::= 'IDENTIFIER_KEY'
 identifier_segment ::= 'IDENTIFIER_KEY' terminal_symbol
 eol_segment ::= 'EOL_KEY'
 eol_segment ::= 'EOL_KEY' terminal_symbol
 eof_segment ::= 'EOF_KEY'
 eof_segment ::= 'EOF_KEY' terminal_symbol
 terminal_symbol ::= 'SYMBOL'
 terminal_symbol ::= 'MACRO_NAME'
 alias_segment ::= 'ALIAS_KEY'
 alias_segment ::= alias_segment 'ERROR_KEY' produces alias_rhs
 alias_segment ::= alias_segment 'EOL_KEY' produces alias_rhs
 alias_segment ::= alias_segment 'EOF_KEY' produces alias_rhs
 alias_segment ::= alias_segment 'IDENTIFIER_KEY' produces alias_rhs
 alias_segment ::= alias_segment 'SYMBOL' produces alias_rhs
 alias_segment ::= alias_segment alias_lhs_macro_name produces alias_rhs
 alias_lhs_macro_name ::= 'MACRO_NAME'
 alias_rhs ::= 'SYMBOL'
 alias_rhs ::= 'MACRO_NAME'
 alias_rhs ::= 'ERROR_KEY'
 alias_rhs ::= 'EOL_KEY'
 alias_rhs ::= 'EOF_KEY'
 alias_rhs ::= 'EMPTY_KEY'
 alias_rhs ::= 'IDENTIFIER_KEY'
 start_segment ::= 'START_KEY'
 start_segment ::= start_segment start_symbol
 headers_segment ::= 'HEADERS_KEY'
 headers_segment ::= headers_segment headers_action_segment_list
 headers_action_segment_list ::= action_segment
 headers_action_segment_list ::= headers_action_segment_list action_segment
 ast_segment ::= 'AST_KEY'
 ast_segment ::= ast_segment action_segment
 globals_segment ::= 'GLOBALS_KEY'
 globals_segment ::= globals_segment action_segment
 trailers_segment ::= 'TRAILERS_KEY'
 trailers_segment ::= trailers_segment action_segment
 start_symbol ::= 'SYMBOL'
 start_symbol ::= 'MACRO_NAME'
 rules_segment ::= 'RULES_KEY' action_segment_list
 rules_segment ::= rules_segment rules
 rules ::= 'SYMBOL' produces rhs
 rules ::= 'SYMBOL' 'MACRO_NAME' produces rhs
 rules ::= 'SYMBOL' 'MACRO_NAME' 'MACRO_NAME' produces rhs
 rules ::= rules '|' rhs
 produces ::= '::='
 produces ::= '::=?'
 produces ::= '->'
 produces ::= '->?'
 rhs ::= //$Empty
 rhs ::= rhs 'SYMBOL'
 rhs ::= rhs 'SYMBOL' 'MACRO_NAME'
 rhs ::= rhs 'EMPTY_KEY'
 rhs ::= rhs rhs_action_segment_list
 rhs_action_segment_list ::= action_segment
 rhs_action_segment_list ::= rhs_action_segment_list action_segment
 action_segment ::= 'BLOCK'
 types_segment ::= 'TYPES_KEY'
 types_segment ::= types_segment type_declarationlist
 type_declarationlist ::= type_declarations
                        | type_declarations 'BLOCK'
 type_declarations ::= 'SYMBOL' produces 'SYMBOL'
                     | type_declarations '|' 'SYMBOL'
 dps_segment ::= 'DISJOINTPREDECESSORSETS_KEY'
 dps_segment ::= dps_segment 'SYMBOL' 'SYMBOL'
 names_segment ::= 'NAMES_KEY'
 names_segment ::= names_segment name produces name
 name ::= 'SYMBOL'
 name ::= 'MACRO_NAME'
 name ::= 'EMPTY_KEY'
 name ::= 'ERROR_KEY'
 name ::= 'EOL_KEY'
 name ::= 'IDENTIFIER_KEY'
 END_KEY_opt ::= //$Empty
 END_KEY_opt ::= 'END_KEY'
 action_segment_list ::= //$Empty
 action_segment_list ::= action_segment_list action_segment

Initial code to generate EBNF understood by https://www.bottlecaps.de/rr/ui:

void Grammar::GenEBNF(void)
{
    //
    // First, flush any data left in the report buffer.
    //
    FILE *f = option -> syslis;

    //
    //    Print the Rules
    //
    fprintf(f, "\n\n//EBNF to be viewed with https://www.bottlecaps.de/rr/ui\n\n");
    {
        int alternate_space = 0;
        char64_t buf;

        //
        // Print the user specified rules.
        //
        for (int rule_index = start_symbol.Length(); rule_index <= num_rules; rule_index++)
        {
            int source_index = rules[rule_index].source_index;
            if (rules[rule_index].IsAlternateProduction())
            {
                for (int i = 0; i < alternate_space; i++)
                    putc(' ', f);
                putc(option -> or_marker, f);
            }
            else
            {
                const char *lhs_name = lex_stream -> NameString(parser.rules[source_index].lhs_index);
                alternate_space = strlen(lhs_name) + (rules[rule_index].IsArrowProduction() ? 3 : 4);
                DisplaySymbol(lhs_name);
                int classname_index = parser.rules[source_index].classname_index,
                    array_element_type_index = parser.rules[source_index].array_element_type_index;
                if (classname_index != 0 && array_element_type_index == 0)
                     fprintf(f, "%s", lex_stream -> NameString(classname_index));
                else if (classname_index == 0 && array_element_type_index != 0)
                     fprintf(f, "%c%c%s", option -> macro_prefix,
                                                         option ->macro_prefix,
                                                         lex_stream -> NameString(classname_index));
                else if (classname_index != 0 && array_element_type_index != 0)
                     fprintf(f, "%s%c%s", lex_stream -> NameString(classname_index),
                                                         option ->macro_prefix,
                                                         lex_stream -> NameString(array_element_type_index));
                else assert (classname_index == 0 && array_element_type_index == 0);

                if (rules[rule_index].IsArrowProduction())
                     fprintf(f, " ::= /*->*/ ");
                else fprintf(f, " ::=");
            }

            for (int j = lex_stream -> Next(parser.rules[source_index].separator_index);
                     j < parser.rules[source_index].end_rhs_index;
                     j = lex_stream -> Next(j))
            {
                if (lex_stream -> Kind(j) == TK_SYMBOL) {
                     VariableSymbol *rhs_symbol = GetSymbol(j);
                     int image = (rhs_symbol ? AssignSymbolIndex(rhs_symbol) : 0);
                     if (image && IsTerminal(image))
                         fprintf(f, " '%s'", lex_stream -> NameString(j));
                     else
                        DisplaySymbol(lex_stream -> NameString(j));
                }
                else if (lex_stream -> Kind(j) == TK_MACRO_NAME)
                     fprintf(f, " %s", lex_stream -> NameString(j));
                else if (lex_stream -> Kind(j) == TK_EMPTY_KEY)
                     fprintf(f, " //%c%s", option -> escape, "Empty");
            }

            putc('\n', f);
        }
 
        putc('\n', f); // leave a gap before listing the remaining rules.
    }

    return;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.