i18n support about gherkin2 HOT 11 CLOSED

To keep this simple I think we should have an .rl file (and associated compiled parser) for each i18n language. This can be done by preprocessing a master .rl.erb file to generate all the .en.rl, .no.rl etc files.

This will result in a lot of code, but this is not problematic at all (except that the gem will be a little big). In Cucumber the individual parser(s) will be loaded lazily based on the # language header or the --language option.

In fact, Cucumber used to be this way. I changed it to be done dynamically a while back, mainly to simplify the build process for contributors. It's not a big deal if we go back to a scheme where a new .rl has to be generated from a template if an i18n language is added or modified.

Current Cucumber can't mix languages in one file (3), and there is no need or desire to do so in this project either.

from gherkin2.

Do you have any objections to me adding the beginnings of i18n into the c_language branch (or merging that branch into master and working on it there)? I'd like to take advantage of the ragel directory in c_language.

The plan is to make ragel/i18n which would contain feature_common.*.rl files for each language, and then output the generated ruby files for each language in lib/gherkin/parser/.

from gherkin2.

Mike - go for it!

I think we should still keep the languages.yml file (maybe rename it to i18n.yml). It's much easier for average Joe to edit a YAML file than a .rl file. Then generate .rl files from an ERB template. WDYT?

Another important thing to keep in mind is space. Some of the i18n languages don't use a space between the Step keywords and the text following it. (I think Chinese is one of them). I noticed that a recent change had hardcoded those spaces in the .rl file. Maybe that's ok, as long as we generate the .rl files with or without the space based on whether or not it's a "spaced" language.

from gherkin2.

I never intended to get rid of the languages file... I'm not that mean. :-) I still have an ancient copy of Cucumber lying around (0.1.something), from which I'll grab the templating tasks. I'll work the spaced languages out with Greg. Shouldn't be hard.

from gherkin2.

I cleaned up feature_common quite a bit. The top few lines have the language specific definitions in all caps, and are the values that can be expected to change for each language version.

I don't see a problem in using ERB to determine what the keyword matches look like based on whether the space is required or not. (space_after_keyword)

It should be just as easy, actually, to let the colon delimiter vary per language as well. I'm not sure if there's ever been any requests for that or not...I imagine the symbol is pretty universal, but wouldn't be surprised if there were better alternatives depending on the language written.

from gherkin2.

There is, I think, another way of dealing with multiple language support, though it might be too clever.

Check out my i18n branch for an example. It should be possible to use feature_common.rl.erb and feature.rb.rl.erb template to generate only 2 files, and accept a language option to the scan method to determine which language to parse against.

The magic is in the scan method of feature.rb.rl, which looks up a machine by name and manipulates the ragel variables to point to the proper one.

The downside to this, I think, would be the generation of a rather large class in Ruby or C that contains a separate machine for each language.

from gherkin2.

I just merged the beginnings of i18n support into my master. If you check it out you'll need to run "rake ragel:i18n" the first time before running rake spec, or the i18n spec will fail (it looks for the Norwegian parser). rake spec will generate the English parser every time it runs.

There's still work to do: I'm going to cleanup the rake tasks some more, particularly to break up the generation of ruby from erb via rl into phases, rather than going straight to ruby without a chance to examine the rl like it does currently. The i18n spec is bare bones at the moment, (one pending spec that itself half-works), so there's lots of room for improvement there, too.

I also added a module method to Gherkin::Parser that you can use to retrieve a parser for a particular language. I'm not sure if it's worth anything at all, but my goal at the time was to get the specs back up and running rather than winning the brilliant Ruby award. I'd be interested in suggestions about that one for sure.

from gherkin2.

Oh yeah, space after keyword support is still missing. Shouldn't be hard to add to the erb though.

from gherkin2.

After looking into it further, I like the one file per language method Mike implemented a lot better. I managed to get things working with a single feature_common file and the ragel itself responsible for i18n support, but it's noticeably slower, which suggests throwing it out.

That method results in a gherkin taking up half as much space as with a file for each language, but feature_common.rb weighs in at a whopping 600K.

from gherkin2.

The current implementation of Parser#[] only supports one natural language per run. While I don't think anyone will use several natural languages in practice, I still think Gherkin should support several languages in one process.

from gherkin2.

I just pushed support for languages that don't require spaces after bare keywords. Add that to the multi-languages per process support, and I think this ticket is done.

from gherkin2.