Is it possible to write a grammar to parse the following pattern: <c

Question regarding grammar about java-probabilistic-earley-parser HOT 2 CLOSED

digitalheir commented on June 15, 2024

Question regarding grammar

from java-probabilistic-earley-parser.

Comments (2)

digitalheir commented on June 15, 2024

It depends. Your question is a bit underspecified. It depends how well-defined your rules actually are.

Try something like this.

final Terminal<String> uninterestingTerminal = new Terminal {
 	... implement what terminals are non-interesting ...
}

private static final Grammar grammar = new Grammar.Builder("myGrammar")
            .addRule(
                    NotInteresting,

		    uninterestingTerminal, 
            )
            .addRule(
                    NotInteresting,
		    
                    NotInteresting,
		    uninterestingTerminal
            )

            .addRule(
                    RULE1,

                    terminal1,
		    terminal2
            )
            .addRule(
                    RULE2,

                    terminal3,
                    terminal4
            )
            .addRule(
                    S,

                    NotInteresting,
                    RULE1,
                    NotInteresting,
                    RULE2
            )
            .build();

Note that the added value of this library is the probabilistic part. If you don't need to do probabilistic parses, you probably better use some other context-free grammar or parser combinator.

from java-probabilistic-earley-parser.

gembin commented on June 15, 2024

Just a simple example: i want to match any DIGIT4 in a sentence.

NonTerminal S = Category.nonTerminal("S");
Terminal<String> ANY = new RegexTerminal(".*");
Terminal<String> DIGIT4 = new RegexTerminal("\\d{4}");
Grammar<String> grammar = new Grammar.Builder<String>("test").setSemiring(LogSemiring.get())
			.addRule(1.0, S, ANY, DIGIT4, ANY)
			.build();
Parser<String> parser = new Parser<>(grammar);
// works
parser.getViterbiParseWithScore(S, Tokens.tokenize("anything 1234 others"))

// NOT working
parser.getViterbiParseWithScore(S, Tokens.tokenize("anything 1234 others 4567 bla bla ..."))