Split parser into separate files about lalrpop HOT 9 OPEN

Discussing with @wagenet revealed an additional feature-request for LALRPOP modules: the ability to declare generic parameters on them, so that one can have common code that is included in multiple places, but with slight tweaks.

I am thinking something like this. One can declare a partial grammar like so:

// predicate.lalrpop
#[partial] // this indicates that the grammar is not meant to be used on its own
grammar();

// in a partial grammar, you can declare empty nonterminals:
Atom;

// these can be used just like you would use a macro parameter,
// so they can appear in types and in expressions:
Predicate: Predicate<Atom> = {
    <a:Atom> => Predicate::Atom(a),
    <l:Predicate> "or" <r:Predicate> => Predicate::Or(l, r),
    <l:Predicate> "and" <r:Predicate> => Predicate::And(l, r),
};

Then one can use a #[partial] grammar like so:

use grammar "predicate.lalrpop" where {
    Atom = Identifier
};

...

Identifier: String = ...;

Foo = {
    Predicate, // this will have type Predicate<String>, since Identifier: String
};

These partial grammars would really be treated like macros. That is, we will parse partial grammars in LALRPOP but otherwise ignore them. The macro-expansion phase will replace a use grammar "xyz" by loading the file, parsing it, and expanding it inline, possibly doing various substitutions as it goes.

We could probably include the ability to make various other tweaks; for example, it might be nice if you could modify the names of the things in that grammar in some systematic way to avoid conflicts. e.g. if I have two versions of the same grammar:

// Propositional logic
use grammar "predicate.lalrpop" as P where {
    Atom = Identifier
};

// First-order logic
use grammar "predicate.lalrpop" as F where {
    Atom = FunctionCall
};

both of these would want to define Predicate. By writing as P or as F, I'd be renaming those entries to something like P::Predicate and F::Predicate. But let's leave that for future work.

from lalrpop.

Not currently possible, but I've been wondering how long until somebody asks for it. :) Seems like a good idea. I have to think about how to do that. Shouldn't be too hard to do something.

On Fri, Nov 13, 2015, at 11:34 PM, Saghm Rossi wrote:

Is there any way to import one parser file's nonterminals for use in
another? The .lalrpop file I'm writing has been getting pretty large,
so I was hoping to modularize it a bit. However, I'm not sure how to
go about doing if (or if it's possible about all).

— Reply to this email directly or view it on GitHub[1].

Links:

1. https://github.com/nikomatsakis/lalrpop/issues/42

from lalrpop.

One idea simple I had would be to specify a vector of the names of the lalrpop files to specify the ordering rather than just having process_root, and allowing the nonterminals of a completed file be available to the later ones. I haven't dug very deeply into the code though, so I don't know how feasible this would be.

from lalrpop.

I was thinking of adding a kind of "include" directive, so that one lalrpop file can include others. (These included files would not be processed by the main driver; probably they would not have a grammar declaration at all, actually, so that we know they are not "main" files.) This would certainly be a simple stopgap measure for now. Eventually I'd like to allow a more sophisticated system for importing and evolving grammars, similar to PPG.

from lalrpop.

➕1 As my grammar is growing, it would be nice to start to split it up by concern.

from lalrpop.

Is this implemented yet?

from lalrpop.

@QuentinJanuel nope. But if you want to try it, feel free to give it an attempt and submit a PR :)

from lalrpop.

I'm not familiar enough with Rust macros yet so I don't think I am capable of solving this, but I will still give it a try I guess

from lalrpop.

@QuentinJanuel any luck? I think this is a super awesome feature if it can get implemented!

from lalrpop.