We all know here that JSON5 is a great format for all kinds of project configuration files or simply data annotation.

I think it would be interesting to look at existing, real-world projects that already use a custom-flavored JSON superset and would benefit from adopting JSON5.

I started a gist here: https://gist.github.com/4078082
Feel free to contribute!

I noticed that

JSON5.parse("{ ValidationError: { title: [ [Object] ] } }");

Fails with following message:

SyntaxError {message: "Unexpected 'O'", at: 32, text: "{ ValidationError: { title: [ [Object] ] } }"} 

I know this is not part of ES5, but considering that multiline discussion, it might make sense to break compatibility with JS anyway - who is using eval() these days anyway?

Allowing hexadecimal floating point values in the form of [-]0xh.hhhp[+-]d (see printf(3), conversion specifier aA) would allow it to store and read floating point values without loss of precision. With the current approach to floating point numbers, floating point values internally always have to be converted to/from a decimal representation which always loses precision.

Current the CLI only converts JSON5 files to JSON. It validates the file in the process, so why not just have an option to validate your JSON5 files to see if they have any errors?

I would like JSON5 to have better support for text blocks. Currently, JSON's only way of encoding blocks of text is to enclose them in double quotes. So, any double quotes in the text must be escaped. Also, any embedded newlines are retained.

YAML (yaml.org) allows blocks of arbitrary text, the only restriction being that it must be indented past the key. It also supports two forms of text blocks: ones which begin with a "|" retain all white space (including newlines and tabs), as is; ones which begin with a ">" convert all sequences of white space to single spaces.

Python (python.org) uses a sequence of three double quotes (""") to begin and end a text block. This pretty much resolves the problem of embedded quotes, as this sequence is very unlikely to appear in a normal text block.

Variations on this approach (eg, ""|...|"", ""<...>"") might allow JSON5 to handle white space in the ways that YAML does.

"All of these are optional" sayeth http://json5.org/

This seems bad for interop. Why not make all the features required?

Do you plan to update it? (Currently it haven't support of NaN values...)

And how about support for 'undefined' values and sparsed arrays in JSON5?

As it says. =)

http://about.travis-ci.org/docs/

Seems like it would be useful, but octal numbers in particular might throw people off. I'm wondering if there's a good reason JSON doesn't support it. Not sure.

There are two good ways to deal with them:

1. ES3 way, or treat them as octals, which means:

• \1 -> \u0001
• \8 -> parse error (?)
• 020 -> 16
• 080 -> parse error (?)
• 0o20 -> parse error

2. ES8 way, or treat them as decimals, which means:

• \1 -> 1
• \8 -> 8
• 020 -> 20
• 080 -> 80
• 0o20 -> 16

About strict mode:

Strict mode is not a complete solution, it is just an intermediate mode between es3 code and future ecmascript versions.

And frankly speaking, it failed. The idea was to have a simpler JS parser, but it ended up all JS engines having 2 parsers, and now there are discussions in es-discuss about how can we get "just one JavaScript" back.

But nevermind that. My point is: comparison with strict mode doesn't make any sense, we either look for b/w compatibility or look into the future, and strict mode is just an intermediate one.

What it is and how it is used:

Octal, or base-8 literals, are heavily used in unix programming. Most famous example are file permissions (which are octal based, i.e. you could read 0755 as -rwxr-xr-x):

mode = 0777 & (~process.umask());

(source: substack/node-mkdirp)

Javascript got them from C or something like that, they are rarely used in web. But when node.js came in, octal numbers started to be used a lot (see Brendan's post in es-discuss 2 years ago).

Another use-case is ANSI escape sequences, in unix manuals ESC is usually written as \033:

console.log('  \033[90mrendered \033[36m%s\033[0m', path);

(source: visionmedia/jade)

Why it was considered a bad thing:

Developers who are unfamiliar with language could mistake them for decimal numbers and use them for padding.

Unfortunately, I can't find a single example of such an error right now.

But there is a notable example in es-discuss where people use octals, but probably don't even know they do:

new Date(2014, 08, 05)
new Date(2014, 10, 12)

Here 08 is a decimal, but 05 is octal. But it has no real effect, because single digit numbers are the same in any base.

How ES7+ is going to solve this:

Future javascript versions will allow 0o777 syntax as a direct replacement for 0777. You can test it right now in Firefox console. Unfortunately, I don't know any replacements for inline octals in strings.

Octals aren't deprecated because people don't like them. They are deprecated, because there is a talk about replacing them with decimal numbers, so:

ES3: `0123` -> 83
ES6: `0123` -> parse error
ES8: `0123` -> 123

I use "ES8" loosely, but the idea stays the same.

Now the main question:

Do we stick with legacy ES3 semantics, or follow bleeding edge of ES standards? There is no legacy JSON5 code to worry about, so either choice looks like a valid one.

Just like you can require() JSON files, we should add a hook to let you require() JSON5 files.

See issue #21, specifically @Midar's comment.

The spec seems to imply that only the newline should get stripped, e.g. "The SV of LineContinuation :: \ LineTerminatorSequence is the empty character sequence."

Firefox and Safari strip just the newline, while Chrome/Node (V8) strip all whitespace after the newline as well. Inconsistent. =/

I think JSON5 would benefit from a well-defined grammar like ECMAScript and JSON have. I wrote a prototype. The README is at the bottom.

Per my comment in #19.

If JSON throws an error, JSON5 should throw the same error unless the behavior has been changed by new features of JSON5 over JSON.

We should probably compile a list of all known JSON errors and write tests.

As specified in the first paragraph describing JSON.parse:

JSON ... allows Unicode code points U+2028 and U+2029 to directly appear in JSONString literals without using an escape sequence.

ES5 does not allow those characters in strings unescaped, since it includes those characters in its LineTerminator production.

In order to be backward compatible with JSON, JSON5 must also support those characters in strings and therefore cannot reference any of ES5's productions that recursively reference LineTerminator or LineTerminatorSequence, namely Comment and StringLiteral. At least not without caveats.

For this reason, I have updated the Grammar to indicate that the LineTerminator and LineTerminatorSequence productions should be substituted with alternative versions defined therein.

The alternative was to recursively redefine the very long Comment and StringLiteral productions as JSON5 versions.

No changes to code are required.

npm test doesn't work in Windows

Any instruction to run the tests manually in Windows?

BTW, npm link is not supported in Windows

To test against octal numbers, for example. (To support issue #2)

In current JSON specification I miss a way of expressing positive and negative infinity. Not-a-number is nice to have too, but it is not so important, as it coerces to null (both are falsy). Could it be added here?

[Edited by @aseemk: we've added infinities; so updated this title to reflect that we should add NaN too.]

It'd be great to have a json5 binary to convert .json5 files to .json. This'd be equivalent to e.g. coffee -c.

ES5 supports numeric literals as property names in object literals. [ES5 §11.1.5]

This allows productions like:

{    1 : 'int',
    .2 : 'float',
    3. : 'trailing dec',
  4e-2 : 'exp',
  0x50 : 'hex' }

All property names are normalized to strings. So the above will result in:

{    '1' : 'int',
     '3' : 'trailing dec',
    '80' : 'hex',
   '0.2' : 'float',
  '0.04' : 'exp' }

Note that the definition of numeric literal precludes signed numbers and special IEEE 745 values, so the following productions are either errors or not treated as numeric values.

{        -1 : 'neg (error)',
         +1 : 'pos (error)',
   Infinity : 'inf (valid but parsed as identifier)',
  -Infinity : 'neg inf (error)',
        NaN : 'not a num (valid but parsed as identifier)' }

Are there plans to support the same syntax that ES5 does? I don't see floating points used as property names very often, but I can see decimal and hexadecimal integers being useful.

An object that has a toJSON method uses that method to delegate during a call to stringify(). This is not yet supported in JSON5, but should be.

Since npm is one of the primary use cases for more human-friendly JSON, it'd be fitting if our own package.json was a .json5 file, and we compiled to the .json file that npm (currently) expects. This'd have to be a manual process unfortunately.

See issue #21, specifically @Midar's comment.

Interestingly, looking at the code, I would expect this to get rejected already, but it seems to get accepted, with the backslash ignored. Should create a test case and investigate.

See discussion at pull request #9.

Edit: editing this case to allow these after @Midar's comments below.

Since I already have comments and the number extensions in my JSON parser, I got interested when I found out about JSON5. When I noticed that it supports ', this got me curious while implementing it: How differently is ' handled from " in JS? I tried this in node, and as you can clearly see, it's a mess:

> '\foo'
'\foo'
> '\e'
'e'
> '\\e'
'\\e'

So basically, it's not handled like double quotes - and it also has a bug: It's not possible to write a backslash followed by a a letter other than b, f, n, r or t. \u is handled like in double quotes:

> '\u'
'u'
> '\u123'
'u123'
> '\u0033'
'3'

So a backslash followed by a u is alos not possible.

So, now I looked into the implementation and it seems all it does is set a variable to the delimiter and then look until that delimiter comes again, unescaped, of course. So this clearly is not compatbile to JS!

Now the question is: Is the implementation wrong or the spec? Which one should I implement? The JS behaviour or the behaviour implemented in JSON5? It's either a bug in the doumentation or the implementation.

ES5 allows it. References:

• http://es5.github.com/#x7.6
• https://developer.mozilla.org/en/Core_JavaScript_1.5_Guide/Core_Language_Features#Variables
• http://docstore.mik.ua/orelly/webprog/jscript/ch02_07.htm

Potentially some helpful resources:

• http://stackoverflow.com/questions/147824/javascript-how-to-find-whether-a-particular-string-has-unicode-characters-esp
• http://stackoverflow.com/questions/280712/javascript-unicode

JSON5.parse("{1:true}"); <== error

Edit: the original title of this case was to disallow these things, considered bugs. That was fixed, but now the conversation below asks to consider re-allowing them.

ES5 allows these:

[,]     // => [undefined]
[,null] // => [undefined, null]

But I don't think that's needed in JSON5. Let's keep it simple and closer to JSON here.

This implementation currently allows these (by design); let's fix.

Under ES rules var a = [,,,,] is a five element array, each element being undefined. What would be the expected behaviour of allowing trailing commas in arrays in JSON5? Would it deviate from true ES behaviour and treat the trailing comma as "not actually there", or would it simply allow it and treat it as a new element marker, that element being undefined?

Via @Midar's comment on issue #21:

"foo
bar"

Should be rejected, and it's currently not:

> JSON5.parse('"foo\nbar"')
'foo\nbar'

Should add a test case and fix.

The replacer parameter is not yet supported. Currently, throws an exception, but this should work as in the JSON spec. A good example is on MDN.

Current all strings are double quoted. It would be nice if I can choose between single or double quotes.

Hi,

the Sencha Touch framework uses an auto-generated config file called app.json for its command line tools. It seems json5 can't load the app.json because of an "" in:

"ignore": [
        "\.svn$"
    ],

The console throws a "Bad string" error.

A downside of JSON5 is that it is no longer a full subset of YAML. Both camps made adjustments to ensure this relationship. It would be a shame to loose it for future versions.

Comments are the most obvious issue. YAML uses the # comment marker common to shell scripting, where as JSON5 is using // an /* */. Obviously, JSON5 is using those b/c they are Javascript's comments too. But I am wondering how this difference might be reconciled.

Another potential issue is treatment of \ in single-quoted strings. In YAML, '\n' is the same as '' 'n', while "\n" is a newline. It seems it means a newline in both cases in JSON5.

Other than that I think things are compatible.

According to the note at the bottom of ES5 §15.12.2 regarding JSON.parse:

In the case where there are duplicate name Strings within an object, lexically preceding values for the same key shall be overwritten.

JSON5 throws on duplicate keys because that's what Crockford's implementation does. However, the JSON standard, Node.js, and browsers do not throw on duplicate keys.

> JSON.parse('{"a": true, "a": false}')
{ a: false }

> JSON5.parse('{"a": true, "a": false}')
SyntaxError: Duplicate key "a"

I propose that JSON5 overwrite duplicate keys instead of throwing.

I believe that JSON5 should be completely backward compatible with JSON. Anything that JSON can parse, JSON5 should parse exactly the same. There are other examples where JSON5 is not backward compatible, which I will begin documenting. Perhaps the Wiki is a good place for that?

Are there any ideas on how to support Unicode and their escape sequences in identifiers? What are the options, and what are the pros and cons of those options?

It seems the biggest hurdle is that there is no standard way to determine what Unicode category a code point belongs to. For example, an identifier can start with $, _, or any character in the five Letter categories or in the Number, Letter category. That's 17196 different characters, all spread across the Unicode code point list.

Here are some ideas on perhaps how to solve this:

• Require a dependency that provides support for testing Unicode categories.
• Bake in support for it yourself. I've already done some prototyping and benchmarking.
• Consume Unicode escape sequences and forego testing them against appropriate Unicode classes, willfully violating the ES5 spec.
• Something very obvious I haven't thought of yet.

I think it's worth discussing the costs and benefits of implementing this support. What do you think?

We only test with eval() currently, to ensure that we remain a subset of ES5. But we might explicitly not allow some features of ES5 (like trailing decimal points, see #9) in order to remain closer to JSON. So we should add some test plumbing for features that are valid ES5 but not valid JSON5 (and still valid JSON).

It strikes me how JSON5 moves "JSON" closer to YAML. I'm curious about others thoughts on this.

This, when parsed:

{
    Infinity: Infinity,
    PositiveInfinity: +Infinity,
    NegativeInfinity: -Infinity,
    NaN: NaN,
    PositiveNaN: +NaN,
    NegativeNaN: -NaN,
}

becomes this, when stringified:

{
    Infinity: null,
    PositiveInfinity: null,
    NegativeInfinity: null,
    NaN: null,
    PositiveNaN: null,
    NegativeNaN: null
}

I will create tests when I have the chance.

Most of the methods I've seen for encoding dates and times in JSON seem hacky and non-standard; if you're going to update the standard, doing dates in a nice way would be good.

(I'm not sure what that nice way would be)

Currently, JSON5.stringify just delegates to JSON.stringify. The result does not seem very JSON5 to me. It makes sense that strings produced by JSON5 should be the shortest string that JSON5 can still parse and produce the original object.

So, one enhancement would be to remove quotes around keys that are already valid JS identifiers.

Since this would break compatibility with the builtin JSON object, perhaps a new function could be created (eg- stringify5).

I just hooked up Travis CI support, which is great for testing across multiple versions of Node, and indeed, one of our test cases fails on Node 0.9:

https://travis-ci.org/aseemk/json5/jobs/4405590

The test case is parsing -0xC8, a negative hexadecimal literal. I narrowed the problem down to these lines in the parser:

https://github.com/aseemk/json5/blob/v0.1.0/lib/json5.js#L154-L156

number = +string;
if (!isFinite(number)) {
    error("Bad number");

The problem is that converting the string -0xC8 to a number via the unary + operator used to work, but now returns NaN. This changed from Node 0.8 to 0.9.

I investigated, and indeed, this is a breaking change in V8, but a fix technically:

http://code.google.com/p/v8/issues/detail?id=2240

I'm not sure yet what this means for JSON5, but we have one failing test today on Node 0.9. We don't document that negative hexadecimal literals are supported anywhere, so I think we're fine there, and the parser correctly rejects them, but we should change the test. Not sure how to do this without causing test failures on Node 0.6 and 0.8.

/cc @MaxNanasy FYI =)

I like the railroad diagrams on the JSON site (json.org). I'd like to have these for the JSON5 site, as well.

It is generally a bad idea to have generated file and source file in the same git repository because it's a pain to keep both in sync.

Also, JSON has an ugly syntax, and doesn't support comments. I wonder why people would even think about using that? :)

Besides, some npm forks are able to work with package.json5 files for 2 months now. So you shouldn't even need to compile anything. :P

// ref #13

I want to port JSON5 to another programming language.
Is there a portable test data set?

I'm slowly pushing json5 to production systems in a backward compatible manner (which means strict json on the output, expecting json5 on the input).

Since we do expect json5 on the input, we should recognize its mime type as well. Sadly, there is none.

So the question is: what mime type are you planning to register/use in the future?

Since one of the goals of JSON5 is to be more easily writable for humans, and since I happened to find this project by Googling "json trailing comma", I think it makes sense for JSON5 to output text that is more easily writable for humans too.

The examples in the README would be output as (note the added trailing commas):

{
    foo: "bar",
    while: true,
    this: "is a multi-line string",
    here: "is another",
    hex: 3735928559,
    half: 0.5,
    delta: 10,
    to: null,
    finally: "a trailing comma",
    oh: [
        "we shouldn't forget",
        "arrays can have",
        "trailing commas too",
    ],
}

{
    name: "json5",
    version: "0.2.0",
    description: "JSON for the ES5 era.",
    keywords: [
        "json",
        "es5",
    ],
    author: "Aseem Kishore <[email protected]>",
    contributors: [
        "Max Nanasy <[email protected]>",
    ],
    main: "lib/json5.js",
    bin: "lib/cli.js",
    dependencies: {},
    devDependencies: {
        mocha: "~1.0.3",
    },
    scripts: {
        build: "./lib/cli.js -c package.json5",
        test: "mocha --ui exports --reporter spec",
    },
    homepage: "http://json5.org/",
    repository: {
        type: "git",
        url: "https://github.com/aseemk/json5.git",
    },
}

Of course, this would only be when the space argument is specified with a truthy value. Otherwise, JSON5 should probably output text as compact as possible.

Thoughts?

This project makes JSON harder to parse and maintain because introduces a lot of ambiguity (like traling commas, quoted / unquoted properties, etc) and kills the purpose of JSON (which should be language agnostic and fast to parse). Are you aware of this?

> node -i

> JSON5 = require('json5')
{ parse: [Function], stringify: [Function] }

> JSON5.stringify()
undefined

> JSON5
{ parse: [Function],
  stringify: [Function],
  isWord: [Function: isWord] }

I understand the rationale behind this line of code. But maybe it would be better to move these functions like isWord into their own "helper" module to avoid leaking.

Thoughts?