Error compiling Unicode escape sequence about uglifier HOT 5 CLOSED

I can't replicate this with the same set of gems (although I'm running Ruby 1.9.3-p0) :( I tried with different LANG/LC_ALL environment variables but they didn't change anything. Even TinyMCE was minified correctly in my testing.

Uglifier should be able to handle Unicode characters. UglifyJS replaces escaped code points with the real character and saves few bytes the process. This can cause problems if some part of the system isn't able to handle UTF-8.

The only remotely related thing I found out was that MultiJson.decode('"var str=\"„\""') fails when :json_gem is used as MultiJson.engine, but works when yajl-ruby is used.

from uglifier.

hmmm :(

Unfortunately, with yajl-ruby I am also getting an error:

$ irb
irb(main):001:0> require 'uglifier'
=> true
irb(main):002:0> Uglifier.new.compile(File.read("pr1.js"))
MultiJson::DecodeError: lexical error: invalid character inside string.
                       ["ok","var str=\"\""] 
                     (right here) ------^

        from /[...]/gems/multi_json-1.0.4/lib/multi_json/engines/yajl.rb:10:in `parse'
        from /[...]/gems/multi_json-1.0.4/lib/multi_json/engines/yajl.rb:10:in `decode'
        from /[...]/gems/multi_json-1.0.4/lib/multi_json.rb:76:in `decode'
        from /[...]/gems/execjs-1.2.13/lib/execjs/external_runtime.rb:61:in `extract_result'
        from /[...]/gems/execjs-1.2.13/lib/execjs/external_runtime.rb:27:in `block in exec'
        from /[...]/gems/execjs-1.2.13/lib/execjs/external_runtime.rb:40:in `compile_to_tempfile'
        from /[...]/gems/execjs-1.2.13/lib/execjs/external_runtime.rb:26:in `exec'
        from /[...]/gems/uglifier-1.2.1/lib/uglifier.rb:101:in `compile'
        from (irb):2
        from /opt/ruby_1_9_3-r34209/bin/irb:12:in `<main>'
irb(main):003:0>

Could you give me a hint, where is the replacing of escaped code points with the real characters done? Thanks.

from uglifier.

UglifyJS first parses the source to AST and then generates code from it.

https://github.com/mishoo/UglifyJS/blob/master/lib/parse-js.js#L397
https://github.com/mishoo/UglifyJS/blob/master/lib/process.js#L1373

By the way, what JS runtime are you using? I used Node for testing, therubyracer have also worked well for me. I haven't tried with Spidermonkey.

from uglifier.

I was using spidermonkey-1.8.0rc1. As the error goes away when using nodejs-0.4.7, I'd call this issue solved. :-)

Thanks for the help!

from uglifier.

Ok, good to know. I'll try to find out if there's anything which can be
done to make non-ascii characters work with Soidermonkey.

On Friday, January 6, 2012, r-stu31 <
[email protected]>
wrote:

I was using spidermonkey-1.8.0rc1. As the error goes away when using
nodejs-0.4.7, I'd call this issue solved. :-)

Thanks for the help!

---

Reply to this email directly or view it on GitHub:
#24 (comment)

from uglifier.