Hi everyone, I just grab the code and i am trying to use the extractor and compiler to run on every js code, some places we are using Angular and other ones we are using Backbone, jquery...

Now i can use the same grunt task to extract all the strings, but the format of the compiled .po files created by the task nggettext_compile are not suitable for the old jquery strings, there are any way for me to make my own format and use it in the grunt task?!

Thanks in advance.

What i just did to make it work is:

// angular-gettext-tools/lib/compile.js

var formats = {
    angularjs_and_jqueryi18n: {
        addLocale: function (locale, strings) {
            return {
                name: locale,
                strings: strings
            };
        },
        format: function (locales, options) {
            var angularjs_locales = [];
            locales.forEach(function (locale) {
                angularjs_locales.push(
                    'gettextCatalog.setStrings("' + locale.name + '", ' + JSON.stringify(locale.strings) + ');\n');
            });

            var jqueryi18n_locales = {};
            locales.forEach(function (locale) {
                _.extend(jqueryi18n_locales, locale.strings);
            });

            var module = '' +
            '// AngularJS \n' +
            'if (typeof angular !== "undefined") {\n' +
            '    angular.module("' + options.module + '").run(["gettextCatalog", function (gettextCatalog) {\n' +
            '        ' + angularjs_locales.join('\n') +
            '    }]);\n' +
            '}\n\n' +
            '// Jquery-i18n\n' +
            'if (typeof $.i18n !== "undefined") {\n' +
            '    $.i18n.load('+ JSON.stringify(jqueryi18n_locales) +');\n' +
            '}';
            return module;
        }
    },
    javascript: {
        addLocale: function (locale, strings) {
            return '    gettextCatalog.setStrings(\'' + locale + '\', ' + JSON.stringify(strings) + ');\n';
        },
        format: function (locales, options) {
            var module = 'angular.module(\'' + options.module + '\')' +
        ...

Since #49 we support extracting line numbers from JS. But if the JS has been transpiled from something like CoffeeScript, those line numbers won't correspond to the actual source line numbers. The solution is to leverage any existing source maps that are on the JS.

Strings are sorted here: https://github.com/rubenv/angular-gettext-tools/blob/master/lib/extract.js#L272

Please make this optional, as it is much easier to insert translations in translation software from already translated documents when they appear in the same order as in the original document.

This should be an option for the grunt and gulp plugins as well of course.

For now, I'll just locally comment it out, extract and comment it back in :)

See rubenv/angular-gettext#103

Pull requests welcome

I'm using grunt-angular-gettext and I find ejs files are ignored even though I specify *.ejs

ejs shoud be treated as 'html'

I find it very inconvenient because I have to manually modify this file in node_modules.

I hope you can add .ejs support or as many types as possible.

It's just updated to 2.0.0 but I don't see any changes but "Copyright 2015"

Based on the api, I can use gettextCatalog.getString("me", scope, "recipient") in my javascript to add context inside my .po file. I did not see the generated .po file having the entry "msgctxt": "recipient".

There is a newer version of lodash (3.3.1) which improved performance 20-40% overall by better utilizing the JIT in JavaScript engines, using internal helper functions that avoid optimization disqualifications & increase the likelihood of function inlining.

lodash 3.0.0 is 40% (1.40x) faster than lodash 2.4.1.

In v3 also introduced were lazily evaluated chaining for massive performance wins in certain scenarios.

Any change to update to the latest release?

Currently compiling po files that have blank lines before the headers (e.g. language header) gives the following error.

> grunt nggettext_compile:all
Running "nggettext_compile:all" (nggettext_compile) task
Warning: No Language header found! Use --force to continue.

Aborted due to warnings.

As an error it feels fairly misleading.

Example file that does not build:

msgid ""
msgstr ""

"Language: es-es\n"

Example file that does build:

msgid ""
msgstr ""
# <- note the comment line
"Language: es-es\n"

Hi,

I have the following in my HTML page:
<span translate>Total duration [min]</span>

I could extract this string into POT, and have the translated string compiled into JS file. The translations get replaced for all strings, except for strings which have special character like one mentioned above.

i get: [MISSING]: Total duration [min] . Where ever special characters are present, it is failing to replace.
Is there anything i am missing to add?

Thanks in advance.

Hi!

I have found some cases where one would need another keyword being extracted other than 'gettext' from javascript files.

markerName: 'gettext',

Can this be made configurable?

Moving rubenv/grunt-angular-gettext#28 to here.

It would be nice if line numbers were included along with the filename of the message, much like xgettext does. For example:

#: src/foo/bar.html:46
msgid "Hello"
msgstr ""

This doesn't include the comment when extracted:

/// this comment won't be extracted
gettextCatalog.getString('foo');

You have to wrap the string in a call to gettext:

/// this comment will be extracted
gettextCatalog.getString(gettext('foo'));

Hi,

I have 'nggettext_extract' task defined in my grunt file and when i try to execute i get the following error:

Loading "compile.js" tasks...ERROR>> Error: Cannot find module 'lodash.isarray'
Loading "extract.js" tasks...ERROR

Error: Cannot find module 'lodash.isarray'

This was working fine before, i have version 2.1.0 of grunt-angular-gettext.

Can anyone point me how to solve this?

Thanks in advance.

Is it possible to have also extraction from filters which are in strings in JavaScript files? Angular-gettext supports it.

e.g.

app.config(function($stateProvider) {
  $stateProvider.state('welcome', {
     url : '/',
     templateUrl: 'index.html',
     ncyBreadcrumb : {
        label: '{{"Home"|translate}}'
     }
  });
});

Mostly dev changes, but also:

• Updated cheerio
• Updated lodash

Hi!

If we'll try to process such html with angular-gettext-tools:

<input id="title" placeholder="'{{'BlahBlahBlah'|translate}}'"/>
<!-- Yes, we really need such "' quotes -->

It will produce duplicate keys:

• "BlahBlahBlah"
• "{{'BlahBlahBlah"

As you see, in html attribute we have single quotes inside double quotes, and I think that's the problem.

I think, you need to make Regex for html attributes more 'hungry' in exports.js.
For example, by changing this line to start = '^.*';. It works for me, but I don't know about side-effects.

We use json scaffolds in our code configuration and the inclusion of the character code replacement during compilation is causing breaks in our strings because after compilation the string keys are no longer matching.

Consider:

{
  "title" : "Let’s create a profile"
}

After translation we have a .po file that has:

msgid "Let’s create your account"
msgstr "Blah blah blah"

However after compilation we end up with:

{
  "Let’s create a profile" : "Blah blah blah"
}

Not sure if this will happen in all cases but when used as:

<label for="{{component.id}}" ng-bind-html="component.title | translate"></label>

The translation fails because the strings won't match up.

This likely should have been more than a patch version bump, but at the very least we need some way to configure this behavior so we can switch it off. For now we are simply pegging our version to 2.1.10 until this is resolved.

Hi,

I am trying to generate JSON output file using PO files. Below is the configuration i have used:

nggettext_compile: {
all: {
options: {
format: "json"
},
files: [
{
expand: true,
cwd:"../",
src: ["po/*.po"],
dest: 'lang/',
ext: ".json"
}
]
},
},

The PO files are present in folder named: "PO" as mentioned in 'src'. And i want to output the JSON files in "lang" folder which is at the same level as "PO" folder. I somehow cant get this command working. Neither the folder nor JSON files are generated. Can anyone help me with correct configuration?

Thanks.

Our translators seem to have a thing for translating English to English even if they change nothing (maybe they like 100% completion in their tool of choice?).

After extraction, we have something like:

{
    "en-GB": {
        "Home": "Home",
        "Account": "Account",
        "Support": "Support"
    }
}

We could slap the translators and tell them to remove them all, but think a few would still slip by. I think it'd be great if extraction just did a key===value check and ignored the entry in such a case.

Hi Ruben,

Does your extraction tool support HAML. I'm using Rails + HAML for my templates and would of course like to take advantage of your excellent angular-gettext module. If not, I guess one possibility would be to build a task that pre-processes the haml by converting it to plain html. Could you recommend any library that might be up to the task of converting HAML to HTML?

Cheers
J

I have a use-case for this, for example I have default translations which are generated using .extract, but also I have custom translations which can't be extracted from the code and those are added manually (in separate *.po file).
For example, there is list of items which has description where several values need to be interpolated like name or some other value. But, there are some special items which have totally different description and don't fit the pattern, so in order to avoid IFs I just define that special description for that particular item in that separate manually created *.po file (with the key 123.description for example). Currently, if I point .compile to those files only the last one will be generated. So, I have *.po files like this:

• /default/en.po
• /default/zh.po
• /custom/en.po
• /custom/zh.po

After compiling I'll get translations only for the last one that has been taken by the .compile, and I would like to have translations merged, meaning to have translations from /default/en.po and /custom/en.po into one single en.js (or en.json)

Does it make sense to have this support within this plugin, or there should be separate plugin which would deal with this?

At this moment I'm using gulp version and I'm altering output of the .compile in a way that I'm caching outputs and merge them in dedicated function, but it's a bit dirty and I don't like it...

This is the last piece of CoffeeScript. Only thing holding me back is the open list of pull requests.

CC @gabegorelick

Now I can't directly extract text from es6 scripts

I saw there is gulp support in README, I can't see guild or doc for it.
Could you add same example or guide for gulp?

We were using quite a few instances of this and found that it wasn't being extracted for translation:

{{ app.showingMore ? 'Show less' : 'Show more' | translate }}

We've had to switch to having two ngIf, but you sometimes get a flicker of both strings as state switches (Show lessShow more):

<button>
  <span ng-if="app.showingMore" translate>Show less</span>
  <span ng-if="!app.showingMore" translate>Show more</span>
</button>

Not sure if this ever worked, but guess it would be a nice thing to support.

(Tried searching for another ticket but couldn't find one. It seems a similar ticket was created here: angular-translate/grunt-angular-translate#48)

This is a feature request, sorry I don't know how to mark this issue as a feature-request...

Would it be possible/feasible/reasonable (choose one :-) to make 'gettextCatalog' and 'getString' in 'lib/extract.js' to be parametrical?
I think about an option in the grunt task, like:

nggettext_extract: {
  ...
  options: {
    aliasFor_gettextCatalog: 'I18N',
    aliasFor_getString: 't',
  }
  ...
}

The reason is I have a factory to collect all I18N methods (named 'I18N'... :-).
I tought it would be nicer to spread all over controllers code something like

I18N.t('my string')

  or even

I.t('my string')

  instead of

gettextCatalog.getString('my string')

...

Sorry, closing myself after reading a closed issue and better understanding annotations usage... :-(

Is it possible to have also context extraction from gettext() function? I have this situation:

• I have to use context for the label
• The label is defined inside factory (instance of this factory can have five statuses and these are their names)
• Factory is initialized before the gettext catalog is loaded - if I use gettextCatalog.getString() inside this factory the labels might not be translated on view despite of usage the translate filter in view
• I'd like to use gettext() in the factory instead and then the context definition in view in filter but the problem is that the string isn't extracted from gettext() function with its context

Is there any other way how to do this?

For us some of the language files are separated (at least in the dev environment), but can contain the same word in different context.
Currently the last loaded file overwrites the rest for the same word.
Since it's rather a small change I just copy it here, rather then creating a pull request. Thanks!

The catalog->setStrings function

                // Expand single strings for each context.
                for (var context in val) {
                    var str = val[context];
                    val[context] = angular.isArray(str) ? str : [str];
                }
                this.strings[language][key] = val;

probably would be better like this:

                if (!this.strings[language][key]) {
                    this.strings[language][key] = {};
                }
                // Expand single strings for each context.
                for (var context in val) {
                    var str = val[context];
                    this.strings[language][key][context] = angular.isArray(str) ? str : [str];
                }

The extractor currently finds filter expressions using regex. This has several drawbacks:

• It leads to false positive (e.g. {{"Hello"|lowercase}} {{"Second"|translate}} will cause Hello"|lowercase}} {{"Second to be found as translatable string
• It does not recognize expressions expressions if they are slightly more complex. E.g. {{ foo ? ('foo'|translate) : ('bar'|translate)}} will go unnoticed
• It is hard to read. The regex currently says new RegExp(start + '\\s*(\'|"|"|')(.*?)\\1\\s*\\|\\s*translate\\s*(' + end + '|\\|)', 'g'); which is hard to read even though it is still pretty simple. It will get much harder when it is extended with further options such as contexts and comments or when trying to do something about the previously mentioned problems.

Thus I explored other options and had a look at how angular is parsing these expressions. And they are using a small recursive descent parser with the manageable amount of 16 node types. Using their parser would put us in a state where there will be no trouble with comparing syntaxes between gettext's parsers and the original.

Unfortunately the angular guys did not seem to anticipate that someone else but them will use their parser. But with a little bit of hacking it is possible. I created a proof of concept here: https://github.com/yankee42/angular-gettext-tools/blob/extract-using-angular-ast/test/parse.js

It is actually quite easy except of the hacks I needed so that angular runs on nodejs without browser and the hack I needed to get hold of AST and Lexer. I am no nodejs expert so maybe someone can come up with a better idea? Possibly angular's parse.js file could be used stand alone, but this requires figuring out which files are all dependencies and need to be included as well. My current approach at least has the advantage that the parser can be kept up to date with npm.

If you think that this is interesting I'd try to actually integrate this into gettext.

Could you support xml as extracting files?

It would be nice if Extractor could extract strings when you use the translate filter, instead of just the translate directive. The example from the docs:

<input type="text" placeholder="{{'Username'|translate}}" />

would then result in 'Username' being added to the POT.

Obviously this would only work for string literals passed through the translate filter, e.g.

<input type="text" placeholder="{{'User' + 'name'|translate}}" />

won't work.

I know the extractor supports the externalizing the translate-context for the html file. Does it have the corresponding support for the javascript file?

Based on the api, I have gettextCatalog.getString("me", scope, "recipient") in my javascript. I did not see the generated .po file having the entry "msgctxt": "recipient".

Locally, I can make the extractor work for the js file by modifying the walkJs of Extractor.prototype.extractJs method.

if (isGettext(node) || isGetString(node)) {
    str = getJSExpression(node.arguments[0]);
    context = node.arguments[2] ? getJSExpression(node.arguments[2]) : null;
} else if (isGetPlural(node)) {
    singular = getJSExpression(node.arguments[1]);
    plural = getJSExpression(node.arguments[2]);
    context = node.arguments[4] ? getJSExpression(node.arguments[4]) : null;
}
if (str || singular) {
    var leadingComments = node.leadingComments || (parentComment ? parentComment.leadingComments : []);
    leadingComments.forEach(function (comment) {
        if (comment.value.match(/^\/ .*/)) {
            extractedComments.push(comment.value.replace(/^\/ /, ''));
        }
    });

    if (str) {
        self.addString(reference, str, plural, comments2String(extractedComments), context);
    } else if (singular) {
        self.addString(reference, singular, plural, comments2String(extractedComments), context);
    }
}

Can the similar fix be added into the next release?

By using typescript to parse js files we can parse files with typescript or flow type annotations.

I have an implementation in kvantetore/angular-gettext-tools@3130c38, that passes all the tests.

It's all tangled up with #105, but I can create a PR if there is any interest

tl;dr: The following breaks string-extraction: <p translate>{{ n <= 1 ? 'a' : 'b' }}</p>

When I did a quick scan through already-filed issues I couldn't find any similar to this – sorry if I missed it and this is a duplicate.

Steps to reproduction

1. In an html-element with a translate-attribute, use a smaller-than-or-equal-to operator (<=) in a data-bounds ({{ }}) expression.
2. Run the nggettext_extract grunt-task

Expected:

• Only the inner contents of the html-element should be extracted by the grunt-task. ({{ n <= 1 ? 'a' : 'b' }})

Actual:

• Everything after the opening tag of the html-element is extracted (the extracted string might look something like {{ n <= 1 ? 'a' : 'b' }}</p></div></section>)

Tests/code

The following will break on the 4th <p>:

<!-- test/fixtures/shorthand_if.html -->
<html>
    <body>
        <p translate>1 start {{ n > 1 ? 'if' : 'else' }} end</p>
        <!-- comment after 1st line -->
        <span>after first</span>
        <p translate>2 start {{ n >= 1 ? 'if' : 'else' }} end</p>
        <!-- comment after 2nd line -->
        <span>after second</span>
        <p translate>3 start {{ n < 1 ? 'if' : 'else' }} end</p>
        <!-- comment after 3rd line -->
        <span>after third</span>
        <p translate>4 start {{ n <= 1 ? 'if' : 'else' }} end</p>
        <!-- comment after 4th line -->
        <span>after fourth</span>
    </body>
</html>

// test/extract_shorthand_if.js

var assert = require('assert');
var fs = require('fs');
var Extractor = require('..').Extractor;
var testExtract = require('./utils').testExtract;

describe('String extractor', function () {
    it('extracts strings with data-bound shorthand-if statements containing smaller-than-or-equal-to operators', function () {
        var files = [
            'test/fixtures/shorthand_if.html'
        ];

        var paragraphs = [
            "1 start {{ n > 1 ? 'if' : 'else' }} end",
            "2 start {{ n >= 1 ? 'if' : 'else' }} end",
            "3 start {{ n < 1 ? 'if' : 'else' }} end",
            "4 start {{ n <= 1 ? 'if' : 'else' }} end"
        ];

        var catalog = testExtract(files);
        assert.equal(catalog.items.length, paragraphs.length);

        for ( var i = 0; i < paragraphs.length; i++ ) {
            assert.equal(catalog.items[i].msgid, paragraphs[i]);
            assert.equal(catalog.items[i].msgstr, '');
            assert.deepEqual(catalog.items[i].references, ['test/fixtures/shorthand_if.html:' + ( 3 + i * 3 )]);
        }
    });
});

Result/output

AssertionError: "4 start {{ n <= 1=\"\" ?=\"\" 'if'=\"\" :=\"\" 'else'=\"\" }}=\"\" end<=\"\" p=\"\">\n        <!-- comment after 4th line -->\n == "4 start {{ n <= 1 ? 'if' : 'else' }} end"
+ expected - actual

-4 start {{ n <= 1="" ?="" 'if'="" :="" 'else'="" }}="" end<="" p="">
-        <!-- comment after 4th line -->
-        <span>after fourth</span>
-    </=>
+4 start {{ n <= 1 ? 'if' : 'else' }} end

Workaround

Use a greater-than operator (>) and swap your ‘if’ and ‘else’ values.

<td data-title="'Name'|translate">{{ user.name }}</td>
<td data-title="'Age'|translate">{{ user.age }}</td>
<td data-title="'Job'|translate">{{ user.job }}</td>

This only extracts "Name" and "Job" (every alternate). If I use {{ }} around the string with the translate filter, then it works, but this breaks in the browser with an angular(or perhaps ngTable) error.

Currently the extractor only looks for gettext() in Javascript files, but when using Typescript or any class based approach, gettext is injected in the constructor and bound to a property. The problem is this.gettext gets ignored by the extractor.

Hi,

I have the following in my HTML page:
< span translate>Total duration [min]</span>

I could extract this string into POT, and have the translated string in compiled JS file. The translations get replaced for all strings, except for strings which have special character like one mentioned above.

i get: [MISSING]: Total duration [min] . Where ever special characters are present, it is failing to replace.
Is there anything i am missing to add?

Thanks in advance.

Given two consecutive attributes in a DOM node, only the first one gets extracted.

For example in the following div only string1 will get extracted:

<div attribute1="'string1'|translate" attribute2="'string2'|translate">
</div>

However if there is a non matching attribute between, it works as expected:

<div attribute1="'string1'|translate" class="test" attribute2="'string2'|translate">
</div>

This is due to the RegExp used inside extract.js and can be fixed by resetting lastIndex on line 311ff:

str = matches[2].replace(/\\\'/g, '\'');
self.addString(reference(n.startIndex), str);
noDelimRegex.lastIndex = 0;

See rubenv/angular-gettext#103

Pull requests welcome

{{"1 car" | translatePlural:n:"$count cars"}}
{{"1 car" | translatePlural:n:"$count cars":"context"}} // With a context

for reasons not important to this issue, i have my html template inside es6 .js files which export the templates as string

// template.js
export default `
  <h1 translate>Heading</h1>
  <p translate>Text in paragraph</p>
`;

notice the backtick which enables multiline html inside js files

when i run the extractor on these file ([email protected], using [email protected]) the tags from these template files dont get extracted

is it me, or is this a bug?

Hi, this issue was in grunt-angular-gettext, but before that we need to implement it in angular-gettext-tools.

See rubenv/angular-gettext#103

Pull requests welcome

{{"1 car" | translate:"context"}}

One of my views contains:

<div ng-init="page.title = ('Home Page'|translate)"></div>

'Home Page' does not get extracted from the HTML by grunt-angular-gettext.

(If I add it manually to my .po file or translations.js; the web browser displays the translated string properly).

I'm building an angular app that I need to be localized for all the main world cultures.
I have currently identified 74 languages.
I have build an automatic translation tool (which uses Google translate page, since Google translate API's are now a paid service) as an aid for translators.
I will not allow users to choose among all languages at runtime (I just do it during development), but I want to localize my web app to the browser's language.
My problem is that "translations.js" is now bigger than 1Mb, and I'm only beginning the development stage... :-(
Currently I load locales dynamically (with "angular-dynamic-locale"), and I would like to load dynamically translations, too...
Should it be possible - at least in theory - to produce one "translation-$language.js" file per language, and load it dynamically on page load?

Hello!

We added a developer to our team that works on Windows. Unfortunately, each time he runs the extractor, the POT file has all its reference filenames changed to have backslashes instead of forward slashes. It produces hundreds of diffs like,

-#: site/home.html:93
+#: site\home.html:93

And of course, the next time one of our unix-based developers regenerates the file, the changes are all reversed.

Considering those are just references, can the format be normalized?

For now, compile() discards fuzzy entries, but keeps obsolete enties ; this should be the contrary imho, or at least configurable by options (skipFuzzy, skipObsolete...).

Please tell me what the way to go is, I may prepare a PR for it.

angular-gettext shouldn't be in the business of trying to guess whether to use the JS or HTML extractor on a given file. As it stands, we get feature requests to add support for HTML-like languages all the time. We can leave the existing strategies in place, but the docs should be updated to have people just call extractHtml or extractJs directly.

The angular plugin allows using the translate directive as an element. Though the extractor does not extract texts from translate tags, only when it is used as an attribute.

I got code sitting here to expose the msgid instead if no translation was given in the po file yet.
either I keep it in my fork and we use that one instead, but if you are open to a PR i can send it over here :)

it is optional, and default situation does not change. only if you pass in the option it will expose the keys in the compiled javascript file / json file, instead of skipping non translated items.

Hi,

I cannot find the way to maintain the old translations and not overwrite them everytime.

this is my code

gulp.task('translations:parse', function () {
return gulp.src([conf.paths.src + '/app/**/*.html', conf.paths.src + '/app/**/*.js'])
    .pipe(gettext.extract('translations.pot', {}))
    .pipe(gulp.dest(conf.paths.src + '/app/translations/'));
});

So the first time the file "translations.pot" is generated. If I translate directly in this file, and I run gulp translations:parse again, the translations saved in "translations.pot" will be lost.

So let's say I have in translations.pot

#: partials/home.html:3
msgid "Welcome"
msgstr "Bienvenido"

After running gulp translations:parse again, the translations are lost

#: partials/home.html:3
msgid "Welcome"
msgstr ""

Is there any possible solution for that? How can I save the old translations?

Thanks a lot
Adrián Bolonio