andyearnshaw / intl.js Goto Github PK

As per sections 10.3, 11.3, and 12.3, each constructors prototype object should be initialised as an instance itself. To quote the specification for the Intl.NumberFormat Prototype object, section 11.3:

The Intl.NumberFormat prototype object is itself an Intl.NumberFormat instance as specified in 11.4, whose internal properties are set as if it had been constructed by the expression Intl.NumberFormat.call({}) with the standard built-in values of Intl.NumberFormat and Function.prototype.call.

Right now, Intl.js would throw an error if we do this before any locale data is added, so this might need to be done after the first call to Intl.__addLocaleData().

Encountered a situation where RegExp.lastMatch==")" which caused this method to fail due to mismatched parentheses. Easy enough to catch that one case and escape the character but possibly there are other conditions that would cause it to fail so raising it as an issue.

Would be curious to learn the motivation for saving and restoring the RegExp state.

test402 output:

intl402/ch12/12.2/12.2.3_c in non-strict mode: 
--- output ---
/tmp/test262-Tcfvs0.js:289: Test262 Error: Missing component hour from requested subset {"weekday":"long","year":"numeric","month":"numeric","day":"numeric","hour":"numeric","minute":"numeric","second":"numeric"}; locale de-DE.
    throw new Test262Error(message);
          ^

(new Date(1)).toLocaleString('en-US', { year: 'numeric', month : 'numeric', day : 'numeric' });

"31 Dec 1969" instead of "12/31/1969"

test402 output:

/11.3_b in non-strict mode: 
--- output ---
/tmp/test262-RnVreh.js:289: Test262 Error: Calling resolvedOptions on true was not rejected.
    throw new Test262Error(message);
          ^

test402 output:

intl402/ch12/12.1/12.1.1_22 in non-strict mode: 
--- output ---
/tmp/test262-l8xUUM.js:289: Test262 Error: Client code can adversely affect behavior: setter for day.
    throw new Test262Error(message);
          ^

Here is a repro case:

http://jsbin.com/coqoqaci/1/edit

var a = new Intl.DateTimeFormat(['en-US'],  {
        month: 'long',
        year : 'numeric'
    }).format(new Date(1390518044403));

var b = new IntlPolyfill.DateTimeFormat(['en-US'],  {
        month: 'long',
        year : 'numeric'
    }).format(new Date(1390518044403));

where a and b should be equal, they are not:

Native: January 2014

Polyfill: Jan 2014

This issue was reported by Flickr.

=== intl402/ch11/11.3/11.3.2_TRP failed in non-strict mode ===
--- errors ---
uncaught exception: Test262 Error: Formatted value for 123.44500, aa-DJ-u-nu-latn and options {"useGrouping":false,"minimumSignificantDigits":3,"maximumSignificantDigits":5} is 123.44; expected 123.45.
===

This is both partially my fault for being lazy, and Mozilla's fault for not fixing this bug. On my side, for the abstract functions ToRawFixed and ToRawPrecision, I used Number.prototype.toFixed and Number.prototype.toPrecision to cheat. Both those functions have similar rounding bugs in v8 and in SpiderMonkey, but I thought the workaround I added fixed the issue.

The implemented workaround doesn't fix the problem in SpiderMonkey, so this test fails in that environment.

Sample test output:

=== intl402/ch09/9.2/9.2.8_4 failed in non-strict mode ===
--- output ---
/tmp/test262-on41xN.js:1076: Test262 Error: Property length of object returned by SupportedLocales is writable. (Testing with NumberFormat.)
            throw e;
                  ^
===

for es_MX you have:

            "decimal": ",",
            "group": ".",

here is the error:
https://github.com/andyearnshaw/Intl.js/blob/master/locale-data/json/es-MX.json#L886

and it should be the other way around, here is a reference in CLDR:
http://www.unicode.org/repos/cldr/trunk/common/main/es_MX.xml

where you will find:

symbols numberSystem="latn">
decimal draft="contributed">.
group draft="contributed">,

Sample test output:

=== intl402/ch13/13.3/13.3.0_7 failed in non-strict mode ===
--- output ---
/tmp/test262-jgEMhW.js:2259: Test262 Error: Result array element at index 0 should be "3 Jun 2013" but is "undefined, 3 Jun 2013 21:12:40". (Testing with locales undefined; options {"hour12":false}.)
                throw e;
                      ^
===

India, Pakistan, Bangladesh, Sri Lanka, and Bhutan use a special scheme for grouping digits: The last group before the decimal separator has three digits, but all other groups have two digits. For example: 1,00,000.00 (one lakh), 1,00,00,000.00 (one crore).

CLDR specifies this in the decimal pattern strings for locales such as en-IN: #,##,##0.###.

Intl.js doesn't seem to support this grouping scheme.

test402 output:

=== intl402/ch08/8.0/8.0_L15 failed in non-strict mode ===
--- output ---
/tmp/test262-_cYvQH.js:289: Test262 Error: The Collator property of this built-in function must not be enumerable.
    throw new Test262Error(message);
          ^
===

saw a commit message with "Recompiled data using CLDR v24" but README has "CLDR version 23.1" under the "Locale Data" section

Running on Chrome (with native Intl support in CST), the following core formats the time of Jan 1, 2013 1:01:01 PM correctly as 1:01 PM

(new Date (2013,0,1,13,1,1)).toLocaleTimeString("en-US", { hour : "numeric", minute : "2-digit",  hour12 : true } )
"1:01 PM"

But, using Intl.js in both Firefox 26.0 and Safari 7.0.1 I get 7:01 PM

(new Date (2013,0,1,13,1,1)).toLocaleTimeString("en-US", { hour : "numeric", minute : "2-digit",  hour12 : true } )
"7:01 PM"

Also affects toLocaleString

(new Date (2013,0,1,23,59,59)).toLocaleString("en-US")
"Wed, Jan 2, 2013, 5:59:59 AM"

what's the difference between intl.js and ibm ecma402?

Thanks, looks good and useful.
But under what licence is this published?

Thank you very much,
Mihai

test402 output:

intl402/ch11/11.1/11.1.1_a in non-strict mode: 
--- output ---
/tmp/test262-_bzYfd.js:289: Test262 Error: RegExp has unexpected property $_ with value de-DE-u-nu-latn.
    throw new Test262Error(message);
          ^

global.Intl = require('intl');
var date = new Date(Date.UTC(2012, 11, 20, 3, 0, 0));
var options = {year: "numeric", month: "short", day: "numeric"};
console.log(new Intl.DateTimeFormat("en", options).format(date));

Expected: Dec 20, 2012
Actual: December 20, 2012

is it possible to show an example of how to detect if Intl is available then use IntPolyfill in nodejs and Browser.
thanks!

Sample test402 output:

=== intl402/ch11/11.1/11.1_L15 failed in non-strict mode ===
--- output ---
/tmp/test262-um8Nwx.js:289: Test262 Error: The length property of a built-in function must not be writable.
    throw new Test262Error(message);
          ^
===

Sample test output:

intl402/ch06/6.2/6.2.3 in non-strict mode: 
--- output ---
/tmp/test262-lEO8kZ.js:1076: Test262 Error: For de-DD got de-DD; expected de-DE. (Testing with NumberFormat.)
            throw e;
                  ^

Jshint is reporting that some of them have invalid characters:

• npm install jshint
• ./node_modules/.bin/jshint locale-data/json/*.json

It's reporting errors in the following files:
locale-data/json/ar-001.json
locale-data/json/ar-AE.json
locale-data/json/ar-BH.json
locale-data/json/ar-DJ.json
locale-data/json/ar-DZ.json
locale-data/json/ar-EG.json
locale-data/json/ar-EH.json
locale-data/json/ar-ER.json
locale-data/json/ar-IL.json
locale-data/json/ar-IQ.json
locale-data/json/ar-JO.json
locale-data/json/ar-KM.json
locale-data/json/ar-KW.json
locale-data/json/ar-LB.json
locale-data/json/ar-LY.json
locale-data/json/ar-MA.json
locale-data/json/ar-MR.json
locale-data/json/ar-OM.json
locale-data/json/ar-PS.json
locale-data/json/ar-QA.json
locale-data/json/ar-SA.json
locale-data/json/ar-SD.json
locale-data/json/ar-SO.json
locale-data/json/ar-SY.json
locale-data/json/ar-TD.json
locale-data/json/ar-TN.json
locale-data/json/ar-YE.json
locale-data/json/ar.json
locale-data/json/fa-AF.json
locale-data/json/fa-IR.json
locale-data/json/fa.json
locale-data/json/kn-IN.json
locale-data/json/kn.json
locale-data/json/ml-IN.json
locale-data/json/ml.json
locale-data/json/ps-AF.json
locale-data/json/ps.json
locale-data/json/si-LK.json
locale-data/json/si.json
locale-data/json/to-TO.json
locale-data/json/to.json
locale-data/json/uz-Arab.json

Sample test output:

intl402/ch12/12.1/12.1.1_TDTO in non-strict mode: 
--- output ---
../Intl.min.js:11: TypeError: Cannot read property '5' of undefined
?:narrow|short|long)$/.test(f))switch(p){case"month":fv=ca.months[f][tm["[["+p
                                                                    ^
TypeError: Cannot read property '5' of undefined
    at FormatDateTime (../Intl.min.js:11:17981)
    at Date.defineProperty.value (../Intl.min.js:11:29711)
    at /tmp/test262-40Birl.js:2251:47
    at Array.forEach (native)
    at /tmp/test262-40Birl.js:2250:15
    at Array.forEach (native)
    at testWithToLocale (/tmp/test262-40Birl.js:2249:13)
    at /tmp/test262-40Birl.js:2278:1

Intl.complete.js is too bloated to serve to clients, which are unlikely to need data for any locale other than the one defined by their language setting. On the flip side, providing Intl.min.js without locale data introduces the following complications:

• Intl.js requires a special case where the developer needs to check for lack of native Intl and then add the required data either in a blocking manner or asynchronously, adding extra hurdles not necessary with native Intl.
• The client code needs to ensure data is available on the server for the specific locale and provide a fall back to the language otherwise. This could result in multiple requests to the server or writing complex server-side code to handle fallbacks. Likewise, the client may need to canonicalise tags in some cases.

A potential solution to this problem is to write a build script that will compile each language―and the language's sub tag―data into Intl.min.js, resulting in, for example, Intl.aa.min.js, Intl.af.min.js, ..., Intl.en.min.js. The developer could write the necessary server- or client-side code, either based on the Accept-Language header or navigator.language || navigator.userLanguage, to choose the base Intl.<language>.min.js file accordingly.

One potential issue is with deprecated or mapped language tags. I'm not sure how likely these are to appear in legacy, modern or future browsers.

I did a simple test:
script src="../Intl.complete.js"

then I have:
var lang = "es-MX";
var date = new Date(Date.UTC(2010,1,9,3,0,0));

then I have:
var options = {weekday: "long", year: "numeric", month: "long", day: "numeric"};
document.write(new IntlPolyfill.DateTimeFormat(lang, options).format(date));

and no matter which format I am using it always returns:

"lun, 8 '{day}{weekday}' febrero '{day}{weekday}' 2010"

vs if I use Intl.DateTimeFormat returns:
"lunes 8 de febrero de 2010"

IntlPolyfill is not returning OK data

IntlPolyfill is the Intl.js implementation, while Intl is the Chrome native object.

(new IntlPolyfill.NumberFormat('en-us',{useGrouping:false, style:'decimal',maximumSignificantDigits:15})).format(1000000);
"100000"

(new Intl.NumberFormat('en-us',{useGrouping:false, style:'decimal',maximumSignificantDigits:15})).format(1000000);
"1000000"

Interestingly,

(new IntlPolyfill.NumberFormat('en-us',{useGrouping:false, style:'decimal',maximumSignificantDigits:6})).format(1000000);
"1000000"
(new Intl.NumberFormat('en-us',{useGrouping:false, style:'decimal',maximumSignificantDigits:6})).format(1000000);
"1000000"

Currently, we avoid overriding Intl by defining as IntlPolyfill. However, Number.prototype.toLocaleString, Date.prototype.toLocaleString and friends are still defined by our library and take the place of the native implementations.

We should check that these functions are not already wired up to a native implementation of Intl before defining them ourselves.

readme.md doesn't provide any example

We are interested in using Intl.js in browsers which don't have the built-in Intl APIs and in Node.js.

It's still up in the air on whether Node.js binaries will ship with v8-i18n linked. There's a long discussion with [inaccurate] worries about the size of bundling the required parts of ICU. If this stays a compile-time flag for Node.js, it effectively means that the Intl APIs will not exist in Node.js :(

See: nodejs/node-v0.x-archive#4689 (comment) and nodejs/node-v0.x-archive#6371

I would like to hedge this by being able to use this polyfill as an npm package and run it in Node.js. Would you be on board with adding the umd wrap (or similar) around this code instead of assuming window exists in the runtime? Also would you be on board with publishing this as an npm package?

I am willing to open a PR for this, but I want to make sure that you're on board with this idea before I do so. Thanks!

Todos:

• Don't assume this is running in a browser with window
• Add umd wrapper
• Add package.json and publish to npm

Do you have plans already on how to shim the enhanced TimeZone support?

It looks like there's a problem caused by IE8's partial defineProperty support, which only works with DOM objects and causes Intl to fail with Object doesn't support this action at load time. This can be detected by actually testing it inside a try/catch block:

https://gist.github.com/acdha/8810743

Unfortunately, this still isn't enough:

new IntlPolyfill.NumberFormat("fr", {style: 'decimal'}).format(123456);

Object doesn't support this property or method

Interestingly, the monkey-patched toLocaleString() does work:

(123456).toLocaleString("fr", {style: 'decimal'});

123 456

In ToRawPrecision, you have these lines:

f = Math.round(Math.exp((Math.abs(e - p + 1)) * Math.LN10)),
m = String(Math.round(e - p + 1 < 0 ? x * f : x / f));

f is very prone to precision error:
// f = Math.round(Math.exp((Math.abs(20)) * Math.LN10))
// 100000000000001000000

This makes m very inaccurate, at least in our targeted precision of 21 digits.
Result is that numbers formatted using SignificantDigits is not accurate after ~10 figures.

In Chrome Console:

(new Intl.DateTimeFormat('en', { year: 'numeric' })).format(1390518044403);
"2014"

In Intl.js

(new Intl.DateTimeFormat('en', { year: 'numeric' })).format(1390518044403);
"1/2014"

I've been experimenting making the source code a little more modular to try and achieve the following results:

• Better organisation of source code
• Easier to extend with missing or newer spec features and proposals
• No need to rebuild Intl.complete.js every time we need to run the tests
• Configurable build process (e.g. build without DateTimeFormat)

I figured this was necessary when I started looking at the timezone stuff, so I've been doing bits here and there when I could. The modularsrc branch is my most recent effort, with the polyfill being fully functional in Node.js but with no current build process. I've probably made it a little more complicated than I originally hoped for, because of how Node's modules work. There are some "circular" requires (e.g. src/intl.js requires src/locale.js which requires src/intl.js to get the Intl object) but I think I can eliminate those.

I was looking into using UglifyJS2's AST tree transformer to build all the files together with optional minification and source maps. Browserify doesn't seem to be appropriate and adds a bit of cruft (including a require() shim).

I'd like to hear some thoughts and feelings on this before proceeding, because it's quite a large change and I may not have taken the best approach so far.

/cc @caridy, @ericf, @drewfish

When I run jshint on the locale-data/jsonp/*.js files I get the following warning for all files: L1:C13105] W033: Missing semicolon.

Chrome is currently failing tests where properties of Array.prototype and Object.prototype are tainted. Unlike #53, which affects the test environment, the actual test is failing with the message:

"{ browserName: 'chrome', version: '31', platform: 'OS X 10.9' }": "unknown error: Client code can adversely affect behavior: setter for 1.",

This issue is specific to Chrome and does not affect v8/Node.js.

test402 output:

=== intl402/ch11/11.1/11.1.1_20_c failed in non-strict mode ===
--- output ---
/tmp/test262-kLqZBd.js:289: Test262 Error: Didn't get correct minimumFractionDigits for currency PYG; expected 0, got 2.
    throw new Test262Error(message);
          ^
===

Seems to be an issue with the regex in getISO4217data.js.

Wrong formats.

having lang = es-ES

Using Intl Object:
var options = {weekday: "long", year: "numeric", month: "long", day: "numeric"};
document.write(new Intl.DateTimeFormat(lang, options).format(date));

returns:
lunes 8 de febrero de 2010

Using IntlPolyfill:
var options = {weekday: "long", year: "numeric", month: "long", day: "numeric"};
document.write(new IntlPolyfill.DateTimeFormat(lang, options).format(date));

returns:
lun, 8 de febrero de 2010

All the week formats, return the same value:

lun, 8 de febrero de 2010

var options = {month: "long"};
document.write(new IntlPolyfill.DateTimeFormat(lang, options).format(date));

returns:
feb 2010

instead of:
febrero

On v8 version 3.21.17 (installed on a macbook via homebrew), Intl.min.js has the following failed tests:

Failed tests
  intl402/ch11/11.3/11.3.2_L15 in non-strict mode
  intl402/ch11/11.3/11.3.3_L15 in non-strict mode
  intl402/ch12/12.3/12.3.2_FDT_7_a_iv in non-strict mode
  intl402/ch12/12.3/12.3.2_L15 in non-strict mode
  intl402/ch12/12.3/12.3.3_L15 in non-strict mode
  intl402/ch13/13.2/13.2.1_L15 in non-strict mode
  intl402/ch13/13.3/13.3.0_7 in non-strict mode
  intl402/ch13/13.3/13.3.1_L15 in non-strict mode
  intl402/ch13/13.3/13.3.2_L15 in non-strict mode
  intl402/ch13/13.3/13.3.3_L15 in non-strict mode

However, when I edit tests/run402 to use Intl.js instead, I get the following failed tests:

Failed tests
  intl402/ch09/9.2/9.2.1_2 in non-strict mode
  intl402/ch09/9.2/9.2.6_2 in non-strict mode
  intl402/ch11/11.3/11.3.2_L15 in non-strict mode
  intl402/ch11/11.3/11.3.3_L15 in non-strict mode
  intl402/ch12/12.3/12.3.2_FDT_7_a_iv in non-strict mode
  intl402/ch12/12.3/12.3.2_L15 in non-strict mode
  intl402/ch12/12.3/12.3.3_L15 in non-strict mode
  intl402/ch13/13.2/13.2.1_L15 in non-strict mode
  intl402/ch13/13.3/13.3.0_7 in non-strict mode
  intl402/ch13/13.3/13.3.1_L15 in non-strict mode
  intl402/ch13/13.3/13.3.2_L15 in non-strict mode
  intl402/ch13/13.3/13.3.3_L15 in non-strict mode

In Chrome, when I format a currency by name, I get

(12345.67).toLocaleString("en-US", { style : "currency", useGrouping: true, currency : "USD", currencyDisplay: "name"})
"12,345.67 US dollars"

Using Intl.js I get undefined in the output

(12345.67).toLocaleString("en-US", { style : "currency", useGrouping: true, currency : "USD", currencyDisplay: "name"}) 
"undefined12,345.67" 

Internet Explorer 11 treats "name" the same as "code" showing USD 12,345.67

Also, a similar bug exists in Firefox.
Bug 866372 - Currency formatting with currencyDisplay="name" results in error

Sample test output:

=== intl402/ch09/9.2/9.2.6_2 failed in non-strict mode ===
--- output ---
/tmp/test262-U5vKTf.js:1076: Test262 Error: Client code can adversely affect behavior: method push. (Testing with NumberFormat.)
            throw e;
                  ^
===

This issue arises anywhere the specification says a List is to be used. We're using an array instead, so it might be an idea to use an Object with numbered properties and a length instead.

In the generated JSON locale files, it looks like Chinese is behaving a bit strangely.

Because the Chinese months are using long, we end up getting something like this when rendering it out:

2014年 六月月 19日 星期四

The '月' character for month is duplicated, when it really should be:

2014年 六月 19日 星期四

Or even better:

2014年 6月 19日 星期四

The month format should probably be switched out to narrow or numeric instead of long instead, or have the additional '月' removed in the formatter here:

https://github.com/andyearnshaw/Intl.js/blob/master/locale-data/json/zh-Hans.json#L33-L39

Repro:

require('intl');
var date = new Date(Date.UTC(2012, 11, 20, 3, 0, 0));
var df = new Intl.DateTimeFormat('zh', { year: "numeric", month: "long", day: "numeric", weekday: "long"});

df.format(date);

Chrome returns:
2012年12月19日星期三

Intl.js returns:
2012年十二月月19日星期三

intl402/ch12/12.3/12.3.2_1_a_L15 in non-strict mode: 
--- output ---
/tmp/test262-O6n6zg.js:289: Test262 Error: Function's length property doesn't have specified value; expected 0, got 1.
    throw new Test262Error(message);
          ^

Sample test402 output:

=== intl402/ch11/11.2/11.2.1 failed in non-strict mode ===
--- output ---
/tmp/test262-u39TsO.js:289: Test262 Error: Intl.NumberFormat.prototype must not be writable.
    throw new Test262Error(message);
          ^
===

In the Sauce Labs testing, some tests fail that do not fail when running the test manually in the given browser. For example:

"{ 
    browserName: 'internet explorer', 
    version: '11', 
    platform: 'Windows 8.1' 
}": "Unable to get element text (WARNING: The server did not provide any stacktrace information)",

The tainted environment is likely to be the cause of these failing tests. The tainted properties define a setter that throws an error, but are defined as configurable and thus could simply be deleted after the test is run.

Not sure the status of the locale db but just noticed that months are being ordered with a string sort rather than a numeric sort. Also, days of the week are ordered based on string sort rather than the canonical ordering.

intl402/ch12/12.3/12.3.3 in non-strict mode: 
--- output ---
/tmp/test262-gyCxc6.js:289: Test262 Error: Property value gregorian is not allowed for property calendar.
    throw new Test262Error(message);
          ^

Intl.js only supports UTC timeZone, but when formatting time, it calls ToLocalTime which does not utilize the timezone passed in as the third parameter.

Changing the time lookups for the returned Record to UTC methods would return the UTC time codes. This should make supporting timezones in the future easier as it would be modifying from the UTC time.

/cc @andyearnshaw @drewfish @caridy

Sample test output:

intl402/ch09/9.1/9.1_b in non-strict mode: 
--- output ---
/tmp/test262-VA1hs1.js:1076: Test262 Error: Locale zh-Hans-CN is supported, but fallback zh-CN isn't. (Testing with NumberFormat.)
            throw e;
                  ^