gettext strings missing after use of case statement in erb about rails HOT 20 CLOSED

I was able to reproduce it.

I'm not really sure how and where to fix this issue.

I printed the source code that is sent from the ERB parser to the Ruby parser (here) and I got this:

First file:

#coding:UTF-8
_erbout = +''; _erbout.<<(( _('jason test 1') ).to_s); _erbout.<< "\n".freeze
;  case 1 ; _erbout.<< "\n".freeze
;  when 2 ; _erbout.<< "\n".freeze
;  end ; _erbout.<< "\n".freeze
; _erbout.<<(( _('jason test 2') ).to_s); _erbout.<< "\n".freeze
; _erbout

Second file:

#coding:UTF-8
_erbout = +''; _erbout.<<(( _('jason test 1') ).to_s); _erbout.<< "\n".freeze
;  case 1
when 2
end ; _erbout.<< "\n".freeze
; _erbout.<<(( _('jason test 2') ).to_s); _erbout.<< "\n".freeze
; _erbout

(I added when 2 to avoid a syntax error when executing the code)

The first Ruby code seems wrong. When it is executed, it breaks after case 1 ; and the second sentence is never executed.

My guess is that the issue is related to the ERB parser! There is not much we can do here or at the GetText gem, sorry about this.

from rails.

It appears that my analysis was correct and that it is a known ERB issue:

• https://gist.github.com/davidphasson/91613
• https://stackoverflow.com/a/5282650/1243212

I close this ticket since it's not really related to this project.

from rails.

(I added when 2 to avoid a syntax error when executing the code)

Sorry about, I was minimizing the test case and I guess I didn't try to execute it.

<%= _('jason test 1') %>
<% case 2 %>
<% when 1 %>foo!
<% when 2 %>bar!
<% end %>
<%= _('jason test 2') %>

The links you provide certainly sound like the issue, but when I execute this code, it seems work fine; output is:

jason test 1
bar!
jason test 2

And yet my app.pot does not contain jason test 2.

from rails.

Your code still contain the issue mentioned in the links.

To fix it, you have 2 solutions:

<%= _('jason test 1') %>
<% case 2
when 1 %>foo!
<% when 2 %>bar!
<% end %>
<%= _('jason test 2') %>

or (the cleanest!):

<%= _('jason test 1') %>
<% case 2 -%>
<% when 1 %>foo!
<% when 2 %>bar!
<% end %>
<%= _('jason test 2') %>

(notice the -%>)

I tested them both and they worked.

from rails.

Yes, I understand my code is in the same format as mentioned in the links, but when I execute it, I don't see any issue in the output. The case statement seems to work. I'm just trying to understand in what way it's broken, as far as what it outputs.

from rails.

I'm not really sure.

I just passed your code as argument into the ERB parser used in GetText and it returned me this Ruby code:

_erbout = +''; _erbout.<<(( _('jason test 1') ).to_s); _erbout.<< "\n".freeze
;  case 2 ; _erbout.<< "\n".freeze
;  when 1 ; _erbout.<< "foo!\n".freeze
;  when 2 ; _erbout.<< "bar!\n".freeze
;  end ; _erbout.<< "\n".freeze
; _erbout.<<(( _('jason test 2') ).to_s); _erbout.<< "\n".freeze
; _erbout

_erbout only contains the first string.

However, with the 2 "fixed" syntaxes, the generated Ruby code works and returns both strings in _erbout:

_erbout = +''; _erbout.<<(( _('jason test 1') ).to_s); _erbout.<< "\n".freeze
;  case 2 
 when 1 ; _erbout.<< "foo!\n".freeze
;  when 2 ; _erbout.<< "bar!\n".freeze
;  end ; _erbout.<< "\n".freeze
; _erbout.<<(( _('jason test 2') ).to_s); _erbout.<< "\n".freeze
; _erbout

I don't know why it doesn't work with GetText but works with your template.

Maybe Rails calls the ERB parser differently? Or do some post-treatment with the parsed Ruby code to avoid this specific error?

from rails.

I think this may be because Rails uses erubi and translation uses Ruby's ERB class.

code = <<~HTML
<%= _('jason test 1') %>
<% case 2 %>
<% when 1 %>foo!
<% when 2 %>bar!
<% end %>
<%= _('jason test 2') %>
HTML
puts ERB.new(code).src

#coding:UTF-8
_erbout = +''; _erbout.<<(( _('jason test 1') ).to_s); _erbout.<< "\n".freeze
;  case 2 ; _erbout.<< "\n".freeze
;  when 1 ; _erbout.<< "foo!\n".freeze
;  when 2 ; _erbout.<< "bar!\n".freeze
;  end ; _erbout.<< "\n".freeze
; _erbout.<<(( _('jason test 2') ).to_s); _erbout.<< "\n".freeze
; _erbout

puts Erubi::Engine.new(code).src

_buf = ::String.new; _buf << ( _('jason test 1') ).to_s; _buf << '
'.freeze; case 2 
 when 1 ; _buf << 'foo!
'.freeze; when 2 ; _buf << 'bar!
'.freeze; end 
 _buf << ( _('jason test 2') ).to_s; _buf << '
'.freeze;
_buf.to_s

from rails.

Nice catch! Rails 5.1+ uses Erubi and before that it was Erubis.

That explains it.

I'm not sure why GetText still uses ERB. Maybe they want to avoid more external dependencies for non-rails projects?

Except this specific case, do you think Erubi could be a real improvement to GetText, or it wouldn't matter much?

from rails.

I don't really know the difference. Here's the commit where Rails switched, which includes their reasoning.

from rails.

Here is some WIP on GetText to change the ERB parser to Erubi : MichaelHoste/gettext@32c4d67

Tests seem OK, I hope that it will not introduce any side-effect. Feel free to test if it solves your specific issue.

from rails.

translation seems to require <= 3.3.7 in gettext, so it's not easy to switch to that fork. I guess I could fork translation too to remove the requirement, but I assume it's there for a reason and there may be compatibility issues.

from rails.

<= 3.3.7 is required because the pluralization rules of some languages have been updated and we still need to decide if we want to support them (fr has a new special plural rule for multiple of 1_000_000 and we find it quite stupid. 1000000 emails or 1000000 d'emails is not worth the extra-work).

But > 3.3.7 will already work without any known issue.

We will officially bump GetText soon since this feature is important to us.

from rails.

This issue was also escalated to ERB: ruby/erb#4

from rails.

We're facing the same issue. Some of our .erb are just missing completely from translation.io.
It would be great if we'd get a warning at least, we could work around the issue by using your fork of get text which uses erubi MichaelHoste/gettext@32c4d67.

I'd also argue to reopen the bug this is definitely a bug in translate gem that should support erb files supported by rails. And would make it faster to figure out while some translations are missing. Or might be even better open a new issue with explaining what kind of constructs doesn't work and potential workaround, so people don't have to read all the conversation related to the bug.

from rails.

Hi @mfazekas,

I'm sorry about this. This issue was the first time we encountered this ERB parsing bug and we guessed that it must almost never happens. Not a lot of developers use these problematic case in their erb files.

Can I ask you which syntax you had a problem with? Is it exactly the same case syntax issue, or another one?

As you can see above, the issue was escalated to ERB and it was considered a wontfix to them: ruby/erb#4

I tried to fix it at the level of ruby-gettext by switching ERB for Erubi in this pull request: ruby-gettext/gettext#91

But there is some resistance from the maintainer, even if all the tests are still passing using Erubi. Maybe it would help if you could contribute to the thread by explaining your issue.

Another solution would be to fork GetText just for this, but I'm not sure it's worth it.

from rails.

@MichaelHoste it seems that 3 files were missing and all of them had case statements, that doesn't look problematic for me.

Some examples:

<% case @package.current_state %>
<% when "received", "needs_special_handling" %>


...

Other:

 <% case @payment.payment_method %>
      <% when "card" %>

from rails.

that doesn't look problematic for me.

I'm sorry if the word was was badly chosen. It's not problematic to use in your code, it's problematic because this syntax is not correctly parsed by the default ERB parser from Ruby, that is also used by GetText.

Using if conditions or using:

<% 
case @package.current_state
when "received", "needs_special_handling"
%>

<% end %>

should make it work.

I hope my branch will get merged into GetText one day, but in the meantime we'll add a "limitations" section in our README.

from rails.

@MichaelHoste thanks,

I don't have much visibility for every details, but my 2ç:
1.) Maybe a GetText PR could be changed so it's only and option to use ERUBI - instead of removing ERB and changing to ERUBI. Then your patch would have much less impact, as default behaviour would not change at all.
2.) At least a warning would be nice from translate, as otherwise it's hard to detect those issues.

from rails.

Thank you for your suggestions @mfazekas.

1.) Maybe a GetText PR could be changed so it's only and option to use ERUBI - instead of removing ERB and changing to ERUBI. Then your patch would have much less impact, as default behaviour would not change at all.

To be honest, I clearly prefer opinionated solutions. Since Erubi seems objectively better, and is also the default .erb parsing library of Rails, that should be possible to always use it instead of ERB in GetText.

But using it as an option could be a great way to move forward with this! I'll investigate this solution 🙂

2.) At least a warning would be nice from translate, as otherwise it's hard to detect those issues.

We could scan all the .erb files to detect those issues.
But my preference would be to take some time to fix it correctly.

In the meantime, I added this section, hoping that it will help people know about this limitation: https://github.com/translation/rails#limitations

from rails.

The new 1.32 version of this gem is now published, and includes the new Erubi parser from gettext.

This issue should now be completely fixed.

Any feedback would be appreciated @JasonBarnabe and @mfazekas.

from rails.