Cirtus value blocks can't accept string expansion about citrus HOT 5 CLOSED

Thanks for reporting this bug. Indeed, the code you post above should work as is. This was a bug in the way the File grammar was parsing Citrus grammar files. I've fixed the bug and pushed the fix in version 2.1.1.

I made the following gist to demonstrate the fix is working. Remember, the match objects you get back from the parser are just strings, so there's no need to call value on your number object if you're just going to put that value in a string.

http://gist.github.com/655733

from citrus.

Thanks so much for the incredible quick turn!!! Citrus is absolutely phenomenal - I'm really enjoying it tremendously. We had been using treetop, but much prefer citrus (so many reasons!).

1. Although I can always pre-process the input (which I currently do), I'd love to see Foo.parse(input, :ignore_case => true) - so that string constants can continue to be readable, e.g:

'foo'      # call with Foo.parse(input, :ignore_case => true)
instead of:
/foo/i

2. I'd love to see some special case support for tokens. I'm not sure how to best express this, but I have a ton of grammars like this:

    rule keyword
            'keyword' space
    end

    rule space
            (' '+) | eol?
    end

    rule eol
            !.        # hack: end of line...
    end

I'm not exactly sure what the token syntax would be, but it would capture the notion that the term had to be "free standing" in some way. Of course, it is doable without any special syntactic sugar, but I seem to see a lot folks doing this over and over. And the eol hack is particularly ugly.

3. When a grammar fails to load (because it has errors), I'm never sure how to locate the cause.. Does Citrus.load set any class variables I can look at to see the last load_error or something? Mostly just for debugging/development... A rake citrus:check would also do.

Citrus absolutely rocks, in any event!!

ps: happy to write tickets for any of these if you would like me to. But I don't mean to be a bother at all.

from citrus.

No problem. I'm glad you're enjoying using Citrus. Let me try and address these one at a time.

1. This would definitely be useful for some cases that I've encountered as well. However, I'm currently unsure what is the best route.

One is the route that you suggest, i.e. setting an :ignore_case flag for a parse. However, this is a bit rigid for my taste. For one, all tokens in a grammar may not need to be case insensitive. Also, it would slow things down considerably to check each rule at parse time for case sensitivity.

The second option would be to create a different type of primitive for grammars. Perhaps a special notation for strings that would indicate they are not case sensitive. This would be more flexible and a little quicker at parse time, but there wouldn't be any way to implement this at the Ruby grammar level (when creating grammars in straight Ruby).

I'm definitely open to ideas on this one.

2. I'm not sure I fully understand your problem here. The eol hack that you are using here is typically used by grammars to specify that there should be no more characters in the input string at that point (i.e. not the "end of the line" but rather the end of the entire input string), which makes perfect sense when you look at it.

3. In the most recent version of Citrus I created an error class called SyntaxError that should give you a pretty detailed description of where exactly the parse failed. What are you seeing when the grammar fails to load? How could it be more detailed?

from citrus.

Thanks for your thoughtful reply. Indeed, I hadn't considered the programmatic/ruby grammar creation case at all. I like the direction you propose: have the case sensitivity be bound to the string, or even the rule level.

My pointing about eol was just that it shows the subtle complexity of defining tokens with trailing whitespace. I did a bad job of communicating that it is likely that: 'keyword' (' '+ | !.) seems like a lot of mechanism for something which might better read token 'foo' or some such. In the old yacc and lex days, the tokenizer would allow the parser to operate above the character level. I'm not sure I'm making any progress in convincing myself citrus should change at all... I apologize for my partially-formed thinking.

on #3, I think I'm not seeing the exception, but I haven't had a grammar error since I was using 2.0.1, so I'm not sure I would have noticed it. Also, I'm using polyglot to load the grammars so it might be swalling the SyntaxError. I'll write a ticket if indeed there is anything worthy of your time here.

All the best,
-Eric

from citrus.

I've added a case-insensitive string literal type in this commit. It basically means that you can write rules like this (using backticks instead of double quotes)

rule match_foo
  `foo`
end

to match any cased "foo" string. I've needed this same functionality in the past, so I'm glad you brought it up.

As far as tokens go, I'd like to let those thoughts simmer for a while before deciding on a course of action to take, if any.

Also, I've removed SyntaxError in favor of ParseError in a recent commit. ParseError provides more information, and I decided there was no need to wrap it in yet another error layer.

from citrus.