simplejson / simplejson Goto Github PK

View Code? Open in Web Editor NEW

1.6K 65.0 336.0 1.37 MB

simplejson is a simple, fast, extensible JSON encoder/decoder for Python

Home Page: https://simplejson.readthedocs.io/

License: Other

Python 61.64% C 38.36%

json python python3 python2 json-parser json-serializer

simplejson's Introduction

simplejson

simplejson is a simple, fast, complete, correct and extensible JSON <http://json.org> encoder and decoder for Python 3.3+ with legacy support for Python 2.5+. It is pure Python code with no dependencies, but includes an optional C extension for a serious speed boost.

The latest documentation for simplejson can be read online here: https://simplejson.readthedocs.io/

simplejson is the externally maintained development version of the json library included with Python (since 2.6). This version is tested with the latest Python 3.8 and maintains backwards compatibility with Python 3.3+ and the legacy Python 2.5 - Python 2.7 releases.

The encoder can be specialized to provide serialization in any kind of situation, without any special support by the objects to be serialized (somewhat like pickle). This is best done with the default kwarg to dumps.

The decoder can handle incoming JSON strings of any specified encoding (UTF-8 by default). It can also be specialized to post-process JSON objects with the object_hook or object_pairs_hook kwargs. This is particularly useful for implementing protocols such as JSON-RPC that have a richer type system than JSON itself.

For those of you that have legacy systems to maintain, there is a very old fork of simplejson in the python2.2 branch that supports Python 2.2. This is based on a very old version of simplejson, is not maintained, and should only be used as a last resort.

simplejson's People

Contributors

Stargazers

Watchers

Forkers

macro theosp singletoned spinlee vsajip bennypk seanjensengrey singingwolfboy netconstructor arigesher billywu yangguilong kini scottkmaxwell jordanm jsalva antiface ahalife eragnew janneku mikedougherty blancomorao ehudokai nicolaiarocci juancarloscruzd sanyaade-webdev 302sk 872409 victorgp shashwatak sgillies joeribekker soundtrack4life towerjoo tombrennan-eaton oscartj git109 chason fakechris hawstein goinnn shakefu help-lazybones gerpsh monkee-ch tadeu emarkmalone roottan schultheisz ramavarsh samyoungnyc dins32 yokotoka edwardrbaker imclab maykinmedia sys-git lilisong ascii1011 sebastiansch ramaseshan lichray juhas poppingtonic mtjdotcom raviteja91 swati730 wufulin chenbk85 hobson whalerock msabramo sarahcodes nett2 lelit zhaorui56 coinpayee arundorairajan aidan-huang danielbaird zjackz coldeasy mindw dslab-epfl ajay05 sporty djowett jbaldachino jeffreywugz jlgray b-rich xumiao pynef joeswhite kbussell semeion peteyan carlboneri rahvar cluo

simplejson's Issues

Default JSONEncoder won't serialize iterators, can't easily be extended.

http://code.google.com/p/simplejson/issues/detail?id=88

Reported by [email protected], Jan 17, 2011
What steps will reproduce the problem?

simplejson.dumps(iter(range(10)))

What is the expected output? What do you see instead?
The iterator should generate a JSON serialized list.

This is a somewhat thorny issue. I have a large dataset which is lazily loaded in via a generator. I'd like to avoid materializing the entire dataset before serializing, ideally by having SimpleJSON consume from my input generator and write to a FD via dump().

The extension mechanism via `default=some_method' doesn't work, because I need to yield individual elements from the seq, not a materialized seq. The existing _iterencode_list() method is totally capable of serializing lazy seqs, but _iterencode() lacks the necessary conditional to invoke it for those types — and the number of possible types is extremely large. Every method in itertools, for example, returns a different type.

I was able to make this work with stdlib's json module, though it was not pretty. I had to subclass JSONEncoder, copy-paste _iterencode() in, and add the necessary conditional to send lazy seqs through _iterencode_list(). This approach is significantly more painful with SimpleJSON, since _iterencode() and friends are now hidden inside _make_iterencode(), where they can't be touched — I'd have to copy-paste the entire method (and its inner methods) to add one more conditional.

I understand that there's going to be some level of heuristic to detect these types, due to the lack of a shared base type for lazy seqs, but is there not some better way to handle this?

intra-package import statements

I have a question regarding simplejson's intra-package import statements. For example, in decoder.py, there is the statement--

from simplejson.scanner import make_scanner

It seems like this prevents simplejson from being used without installing it. If I drop the simplejson package into my project at the location thirdparty/simplejson and use the import statement--

from thirdparty import simplejson as json

then I get the following error at that line of decoder.py:

ImportError: No module named simplejson.scanner

Is there a way to use simplejson in the way that I'm trying without getting that error? If you used relative imports of the form

from scanner import make_scanner  # or
from .scanner import make_scanner

then it seems like this would not be a problem. I know that PEP 8 discourages the use of relative imports, but for the reason above, I've never understood its rationale for packages where the package owner does not have control over how the package will be used. Thanks in advance for your help.

Unencodable dict keys should call .default()

It's plausible that an arbitrary object may be encodeable as a string, so it seems reasonable that an unknown type of dict key would go through default first, and only raise TypeError if the result is still not stringlike.

use json attribute to encode custom objects

I'm currently working on an issue where I'm writing a library that will interface with other moving parts that use simplejson internally. However, I need to be able to pass them a custom object and have that be serialized. Unfortunately, since the json encoding is done within these projects and isn't directly called by my code, I can't pass it a custom encoder.

Pyramid (one of the moving parts I'm using) has a useful function that will use the json attribute to encode a custom object, but the other one I'm using does not have that. While searching around, I found this patch from the code.google.com repo.

It would be awesome if this behavior could be included in simplejson by default. The current behavior of using the output from repr(o) makes it very difficult to have my object serializable by simplejson in an agnostic manner, since I'm not working with the library directly.

JSON decoder reports wrong column number on first line

http://bugs.python.org/issue17225

TypeError: Decimal(...) is not JSON serializable with use_decimal=True

Hi,

I seem to have run into a problem when attempting to encode decimal types. I am running multiple applications on multiple workers using uWSGI. This problem essentially boils down to the use of multiple sub-interpreters.

In uWSGI a work around for this problem is to use the --single-interpreter option but this prevents multiple applications per process.

The problem in simplejson, I believe, is due to the caching of the Decimal type in the speed-ups C extension. This leads to a problem when checking if we have a decimal because the cached type is the same across all sub-interpreters, but the Decimal object compared is different across each sub-interpreter, i.e. the cache type in the extension is the type from the first sub-interpreter that imported simplejson and thus for subsequent sub-interpreters effectively "Decimal != Decimal".

I originally thought this may have been a problem with uWSGI and was pointed to the same issue that existed in psycopg2 < 2.4.3. I have updated psycopg2 and ruled that out as the cause, and deleting the _speedups.so library to force simplejson to use its python-based encoder caused the problem to disappear.

See the following links for more information:

I'm not very familiar with Python's C API unfortunately, otherwise I would have tried to offer you a patch.

I hope this won't be too awkward to fix.

Cheers,

Nick

2.5.1 can't dump json due to Decimal option (tested on os x)

Here are the commands I ran

╭─○─[Wowbagger.local@Darwin]─[~]─[DON'T PANIC]-[ret:0]─[hist:4268]
╰─[:)]─> ⚡ mkvirtualenv json_test                                                                                                                  ▶▶▶▶▶▶▶▶▶▷
New python executable in json_test/bin/python
Please make sure you remove any previous custom paths from your /Users/adam/.pydistutils.cfg file.
Installing setuptools.............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/adam/.virtualenvs/json_test/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/adam/.virtualenvs/json_test/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/adam/.virtualenvs/json_test/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/adam/.virtualenvs/json_test/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/adam/.virtualenvs/json_test/bin/get_env_details
git: Permission denied

╭─(json_test)─○─[Wowbagger.local@Darwin]─[~]─[DON'T PANIC]-[ret:0]─[hist:4269]
╰─[:)]─> ⚡ pip install simplejson                                                                                                                  ▶▶▶▶▶▶▶▶▶▷
Downloading/unpacking simplejson
  Downloading simplejson-2.5.1.tar.gz (52Kb): 52Kb downloaded
  Running setup.py egg_info for package simplejson

Installing collected packages: simplejson
  Running setup.py install for simplejson
    building 'simplejson._speedups' extension
    /usr/bin/llvm-gcc -fno-strict-aliasing -arch i386 -arch x86_64 -O3 -march=core2 -w -pipe -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.2/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.7-intel-2.7/simplejson/_speedups.o
    /usr/bin/llvm-gcc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -isysroot / -L/usr/local/Cellar/readline/6.2.1/lib build/temp.macosx-10.7-intel-2.7/simplejson/_speedups.o -o build/lib.macosx-10.7-intel-2.7/simplejson/_speedups.so

Successfully installed simplejson
Cleaning up...

╭─(json_test)─○─[Wowbagger.local@Darwin]─[~]─[DON'T PANIC]-[ret:0]─[hist:4270]
╰─[:)]─> ⚡ python                                                                                                                                  ▶▶▶▶▶▶▶▶▶▷
Python 2.7.2 (default, Dec  5 2011, 14:52:41) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson as json
>>> json.dumps(json.loads('{"hi":"ho"}'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/adam/.virtualenvs/json_test/lib/python2.7/site-packages/simplejson/__init__.py", line 321, in dumps
    return _default_encoder.encode(obj)
  File "/Users/adam/.virtualenvs/json_test/lib/python2.7/site-packages/simplejson/encoder.py", line 237, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/adam/.virtualenvs/json_test/lib/python2.7/site-packages/simplejson/encoder.py", line 301, in iterencode
    Decimal)
TypeError: make_encoder() takes at most 15 arguments (16 given)
>>>

Make encoding behavior of simplejson possible to override

I would really like to be able to control how a variable is encoded directly. Currently, if a variable is of a type recognized by simplejson it will be encoded automatically, with no imput on my part. I've bumped up against the inability to override atomic encode actions repeatedly, and it has made using this library terribly unpleasant.

Specifically...

I wanted to override encoding so that certain strings would be encoded without surrounding quotes, for code generation. The way the code there is no way to do this short of hacking the module, which is silly.

I want to serialize a graph with cycles. It is really easy to get rid of cycles by ignoring edges to already visited nodes in the graph when moving outward from a visited node. I would love to add this bit of logic, but I can't do that without totally rewriting your class (ironically you already do this with markers, but since I can't get into the code anywhere, that is pretty useless to me).

Ideally, an api for this would let you specify a "translator" function, as well as input/output filters.

simplejson.scanner.JSONDecodeError: Unpaired high surrogate

Hi,

Sorry for taking a while to report this. It took me some time to find the JSON (in about 2.5TB of data) and then I realised I should probably trim it down to the relevant section.

I am unable to parse the following JSON with simplejson 3.1.0 under Linux. This JSON is taken from a result set from ElasticSearch and obviously trimmed down to be as small as possible.

{"a": "\ud8e9"}

I done some testing on a fresh vagrant Ubuntu Precise 64bit virtual machine but previously we have been seeing the error with lucid on EC2.

The error get is;

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/__init__.py", line 398, in load
    use_decimal=use_decimal, **kw)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/__init__.py", line 454, in loads
    return _default_decoder.decode(s)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 393, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Unpaired high surrogate: line 1 column 8 (char 7)

If I disable the speedups with simplejson._toggle_speedups(False) I get a (possibly) slightly more helpful error.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/__init__.py", line 398, in load
    use_decimal=use_decimal, **kw)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/__init__.py", line 454, in loads
    return _default_decoder.decode(s)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 393, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/scanner.py", line 119, in scan_once
    return _scan_once(string, idx)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/scanner.py", line 90, in _scan_once
    _scan_once, object_hook, object_pairs_hook, memo)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 198, in JSONObject
    value, end = scan_once(s, end)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/scanner.py", line 87, in _scan_once
    return parse_string(string, idx + 1, encoding, strict)
  File "/home/vagrant/.virtualenvs/a6e8a75c0b63d8f2/local/lib/python2.7/site-packages/simplejson/decoder.py", line 118, in py_scanstring
    raise JSONDecodeError(msg, s, end)
simplejson.scanner.JSONDecodeError: Unpaired high surrogate: line 1 column 8 (char 7)

However, I'm not totally sure where the blame lies here. I've done a bit of research and this is what I've found.

I mentioned before I only have this problem under Linux. Locally on my mac it works fine. I've not been able to test Windows. I was pointed to this issue by @bigkevmcd: http://bugs.python.org/issue11489 which seems relevant but not conclusive.
I have tested this file with a few other languages for comparison - works fine in Java and Ruby. JavaScript seems to mostly work, although I noticed some display issues in the Chrome console.
I have discovered some other JSON documents cause the same error however they do work when I turn off the speedups. I've not been logging these as I'm turning off the speedups and trying again and then only logging those that still fail. I'll do another run (probably not until Monday now) and add logging of these so I can provide them.
In my tweet I was confused why it worked, failed and then worked after a pip re-install. This was with a different document and because simplejson was originally installed on the server with a package. Then when I pip installed it failed to install speedups - thus I "disabled" them without realising.

So, yeah, that's what I've found. I'm not sure how helpful it is. It does seem that the JSON that work without c speedups (but not with) only are causing a bug but it also seems to me that unpaired surrogate issues should either always work or always not work...

simplejson won't let me override output for NaNs

http://code.google.com/p/simplejson/issues/detail?id=76

Reported by [email protected], Apr 16, 2010
I really want to have a simplejson encoder which lets me override the serialization of NaN objects
to treat them as Nones (ie serialize them to "null")

This is because
a) strict JSON doesn't allow the standard JSON serialization as "NaN" (and one of the JSON
libraries I need to work with doesn't support "NaN")
b) I'm dealing with lots of floating point data, some of which might include NaNs.

If I write a default() function to override simplejson's standard behaviour, it never gets called,
because simplejson only calls default() on a TypeError.

If I switch on allow_nans=False (to put simplejson in strict compliance mode) then the default()
function never gets called because simplejson raises a ValueError, not a TypeError.

And actually, looking at the code, even if a TypeError was thrown, it wouldn't be caught such that
_default would be called.

How best should I do this?

(Obviously I can and currently do work around it by preprocessing data before handing it to
simplejson)

I've just checked the native Javascript implementations of Safari, Firefox and Chrome (using the javascript
shell at www.squarefree.com/shell/shell.html) The following happens in all three (on a Mac, though I doubt it
makes any difference):

JSON
[object JSON]

NaN
NaN

JSON.stringify(NaN)
null

JSON.stringify(NaN) === "null"
true

Add more tests

http://bugs.python.org/issue16559

Documentation doesn't describe what happens when deserialization fails

Documentation doesn't describe what happens when deserialization fails.
As I see from code JSONDecodeError exception is raised on deserialization failures, please document this case.

The 'namedtuple_as_object' kwarg causes backwards incompatibility

Simplejson 2.2 and greater are incompatible with previous versions if you subclass JSONEncoder. By this I mean that simplejson.dumps() calls __init__() with the keyword argument in 2.2+ and without it before then.

This breaks backwards compatibility with, for example, the Python standard library, which contains a modified version of simplejson 2.0.9:

>>> import json
>>> import simplejson
>>> simplejson.dumps({'hello': 'world'}, cls=json.JSONEncoder)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aogier/.virtualenvs/simp/lib/python2.7/site-packages/simplejson/__init__.py", line 334, in dumps
    **kw).encode(obj)
TypeError: __init__() got an unexpected keyword argument 'namedtuple_as_object'

This came up in Django, which has a DjangoJSONEncoder that knows how to handle Django modules. In an effort to support the lowest common denominator of json library (simplejson 1.9 in python 2.6), we stopped subclassing simplejson.JSONEncoder and started subclassing json.JSONEncoder but this breaks simplejson.dumps() due to the way it calls __init__.

django/django@cec6bd5#L5R53

LookupError: unknown encoding: hex

Python 2.6.2

simplejson is no longer working (no changes were made to the server):

>>> import simplejson
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-i686/egg/simplejson/__init__.py", line 111, in <module>
  File "build/bdist.linux-i686/egg/simplejson/decoder.py", line 29, in <module>
  File "build/bdist.linux-i686/egg/simplejson/decoder.py", line 21, in _floatconstants
LookupError: unknown encoding: hex

Different encoding with and without speedups

Hi,

we just ran into an issue with the encodings when speedups are enabled and when they are not.
When speedups aren't enabled a JSON string is always returned as a unicode python string.
With speedups enabled it depends on the JSON input.

With speedups:

type(simplejson.loads('"abc"')) = "str"
type(simplejson.loads(u'"abc"')) = "unicode"
type(simplejson.loads(u'"abc"'.encode("utf-8"))) = "str"
type(simplejson.loads(u'"\xf6"'.encode("utf-8"))) = "unicode"

The same without speedups results in unicode all the time.

compiler warning

on OpenBSD there is a compiler warning while compiling _speedups.c:

cc -DNDEBUG -O2 -pipe -fPIC -O2 -pipe -fPIC -I/usr/local/include/python2.7 -c simplejson/_speedups.c -o /usr/ports/pobj/py-simplejson-3.0.7/simplejson-3.0.7/temp.openbsd-5.3-i386-2.7/simplejson/_speedups.o
simplejson/_speedups.c: In function 'ascii_char_size':
simplejson/_speedups.c:489: warning: comparison is always true due to limited range of data type

might be related that pyconfig.h contains:

#define Py_UNICODE_SIZE 2
#define PY_UNICODE_TYPE unsigned short

relative imports

The source code of simplejson assumes simplejson is in sys.path. In fact it does:

 from simplejson.decoder import ....

This prevents from having two versions of simplejson (one in sys.path and one not in sys.path) because when using the one not in sys.path, the imports would import from the other simplejson (in sys.path) and occasionally cause problems.

Sometimes we have two versions because one is normally easy_installed by a user and another is packaged with an application for portability the app does not assume simplejson is installed so it comes with its own copy).

Can the above imports be rewritten as

 from decoder import ....

? This would solve the problem.

missing entry for README.rst in the MANIFEST

When building RPMs, "python setup.py sdist" needs to produce a valid source distribution, which is not the case

steps to reproduce:

2028 wget http://pypi.python.org/packages/source/s/simplejson/simplejson-2.2.0.tar.gz
2029 tar -xzvf simplejson-2.2.0.tar.gz
2030 cd simplejson-2.2.0/
2031 python setup.py sdist
2032 cd dist/
2033 ls
2034 tar -xzvf simplejson-2.2.0.tar.gz
2035 cd simplejson-2.2.0/
2036 python setup.py build

I'd suggest adding a MANIFEST.in file with this line:

include README.rst

patch for running on Python 2.4

Here is a patch for running on Python 2.4. I know the docs state that it runs on Python 2.5 and later, but this is a very small change that is required for running Sphinx on RHEL5. Hopefully it's useful and will be merged. I'm new to git, so thought I would keep it simple by attaching a diff.

Python 2.4 was raising a SyntaxError and the following change was made to fix this. I'm running simplejson 2.5 from PyPI along with Sphinx 1.1.3. I ran the unit tests with the change and they all pass.

Cheers,
Chris LeBlanc

diff --git a/simplejson/encoder.py b/simplejson/encoder.py
index 05dd05c..5710e67 100644
--- a/simplejson/encoder.py
+++ b/simplejson/encoder.py
@@ -394,10 +394,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
elif value is False:
yield buf + 'false'
elif isinstance(value, (int, long)):

```
           yield ((buf + str(value))
```

                  if (not _bigint_as_string or

                      (-1 << 53) < value < (1 << 53))

                      else (buf + '"' + str(value) + '"'))

           if not _bigint_as_string or (-1 << 53) < value < (1 << 53):

```
               yield buf + str(value)
```
```
           else:
```

               yield buf + '"' + str(value) + '"'
     elif isinstance(value, float):
         yield buf + _floatstr(value)
     elif _use_decimal and isinstance(value, Decimal):

@@ -487,10 +487,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
elif value is False:
yield 'false'
elif isinstance(value, (int, long)):

```
           yield (str(value)
```

                  if (not _bigint_as_string or

                      (-1 << 53) < value < (1 << 53))

                      else ('"' + str(value) + '"'))

           if not _bigint_as_string or (-1 << 53) < value < (1 << 53):

```
               yield str(value)
```
```
           else:
```

               yield '"' + str(value) + '"'
     elif isinstance(value, float):
         yield _floatstr(value)
     elif _use_decimal and isinstance(value, Decimal):

@@ -528,10 +528,10 @@ def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
elif o is False:
yield 'false'
elif isinstance(o, (int, long)):

```
       yield (str(o)
```

              if (not _bigint_as_string or

                  (-1 << 53) < o < (1 << 53))

                  else ('"' + str(o) + '"'))

       if not _bigint_as_string or (-1 << 53) < o < (1 << 53):

```
           yield str(o)
```
```
       else:
```

           yield '"' + str(o) + '"'
 elif isinstance(o, float):
     yield _floatstr(o)
 elif isinstance(o, list):

SyntaxError in simplejson/tool.py with Python 3.3

$ python3.3 -m compileall -f .
Listing '.'...
Compiling './conf.py'...
Listing './scripts'...
Compiling './scripts/make_docs.py'...
Compiling './setup.py'...
Listing './simplejson'...
Compiling './simplejson/__init__.py'...
Compiling './simplejson/compat.py'...
Compiling './simplejson/decoder.py'...
Compiling './simplejson/encoder.py'...
Compiling './simplejson/ordered_dict.py'...
Compiling './simplejson/scanner.py'...
Listing './simplejson/tests'...
Compiling './simplejson/tests/__init__.py'...
Compiling './simplejson/tests/test_bigint_as_string.py'...
Compiling './simplejson/tests/test_check_circular.py'...
Compiling './simplejson/tests/test_decimal.py'...
Compiling './simplejson/tests/test_decode.py'...
Compiling './simplejson/tests/test_default.py'...
Compiling './simplejson/tests/test_dump.py'...
Compiling './simplejson/tests/test_encode_basestring_ascii.py'...
Compiling './simplejson/tests/test_encode_for_html.py'...
Compiling './simplejson/tests/test_errors.py'...
Compiling './simplejson/tests/test_fail.py'...
Compiling './simplejson/tests/test_float.py'...
Compiling './simplejson/tests/test_indent.py'...
Compiling './simplejson/tests/test_item_sort_key.py'...
Compiling './simplejson/tests/test_namedtuple.py'...
Compiling './simplejson/tests/test_pass1.py'...
Compiling './simplejson/tests/test_pass2.py'...
Compiling './simplejson/tests/test_pass3.py'...
Compiling './simplejson/tests/test_recursion.py'...
Compiling './simplejson/tests/test_scanstring.py'...
Compiling './simplejson/tests/test_separators.py'...
Compiling './simplejson/tests/test_speedups.py'...
Compiling './simplejson/tests/test_tuple.py'...
Compiling './simplejson/tests/test_unicode.py'...
Compiling './simplejson/tool.py'...
***   File "./simplejson/tool.py", line 32
    except ValueError, e:
                     ^
SyntaxError: invalid syntax

For compatibility with Python 2.5, you can use:

except ValueError:
    raise SystemExit(sys.exc_info()[1])

Fractions wrongly encoded

Using HEAD version of simplejson:

In [3]: simplejson.dumps([0.3])
Out[3]: '[0.29999999999999999]'

I would expect this to output '[0.3]'. A workaround is to manually turn the fractions into strings before encoding.

Differing output depending on in C speedups are used

I just encountered a glitch in our tests. We have a few tests that depend on the output of simplejson, and we found that simplejson 2.6.0 will return unicode if the C speedups aren't used and str if they are.

With C speedups:

>>> simplejson.loads('"foo"')
'foo'

without

>>> simplejson.loads('"foo"')
u'foo'

It would be preferable if the two modes would produce the same output.

custom JsonEncoder not working on 2.3.2

Looks like JsonEncoder.default is never called

import decimal
import simplejson as json

class JsonEncoder(json.JSONEncoder):
    def default(self, obj):
        print "xxxxx",obj,type(obj)
        return json.JSONEncoder.default(self, obj)

print json.dumps({'a':decimal.Decimal("1.11")}, cls=JsonEncoder)

Output is {"a": 1.11}
It means it is not calling custom Encoder and it seems to encode decimal as float

It is version 2.3.2 installed on Ubuntu 12.04

encoder.py has inconsistent indentation, causing a syntax error in Python 2.4

python

Python 2.4.3 (#1, Feb 22 2012, 16:06:13) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/site-packages/simplejson-2.6.2-py2.4-linux-i686.egg/simplejson/__init__.py", line 112, in ?
    from encoder import JSONEncoder, JSONEncoderForHTML
  File "/usr/lib/python2.4/site-packages/simplejson-2.6.2-py2.4-linux-i686.egg/simplejson/encoder.py", line 400
    if (not _bigint_as_string or
     ^
SyntaxError: invalid syntax

2.3.2 vs. 2.1.1

when using in the Flask with Markup and meet the double quote, The two version's behavior is totally different.
2.3.2 : simplejson.dumps(flask.Markup(u''id="a"')) --> "id=&#34a&#34"
2.1.1 : simplejson.dumps(flask.Markup(u'id="a"')) --> "id="a""
why they are so different, and which one is the RIGHT?

simplejson/_speedups.c:2652: fatal error: error writing to -: Broken pipe

% pip install simplejson
Downloading/unpacking simplejson
  Downloading simplejson-2.1.6.tar.gz (52Kb): 52Kb downloaded
  Running setup.py egg_info for package simplejson
Installing collected packages: simplejson
  Running setup.py install for simplejson
    building 'simplejson._speedups' extension
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c simplejson/_speedups.c -o build/temp.macosx-10.6-universal-2.6/simplejson/_speedups.o
    /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
    Installed assemblers are:
    /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
    /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
    simplejson/_speedups.c:2652: fatal error: error writing to -: Broken pipe
    compilation terminated.
    lipo: can't open input file: /var/folders/MT/MTdESo89GBCaGM7Hevp5+E+++TI/-Tmp-//ccZANzOC.out (No such file or directory)
    ***************************************************************************
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Failure information, if any, is above.
    I'm retrying the build without the C extension now.
    ***************************************************************************
    ***************************************************************************
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Plain-Python installation succeeded.
    ***************************************************************************
Successfully installed simplejson
Cleaning up...

ear

Support for encode option bigint_as_string from scottkmaxwell

https://github.com/scottkmaxwell/simplejson similar to http://php.net/manual/en/function.json-encode.php

DeprecationWarnings in test suite with Python 3.3

When warnings are enabled (using e.g. PYTHONWARNINGS="d" environmental variable), then test suite with Python 3.3 triggers 117 DeprecationWarnings.

$ PYTHONPATH="build/lib.linux-x86_64-3.3" python3.3 simplejson/tests/__init__.py
/tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:52: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val)))
/tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:55: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val, bigint_as_string=True)))
./tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:41: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val)))
/tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:44: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val, bigint_as_string=True)))
./tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:19: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val)))
/tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:22: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val, bigint_as_string=True)))
./tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:30: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val)))
/tmp/simplejson-3.0.4/simplejson/tests/test_bigint_as_string.py:33: DeprecationWarning: Please use assertEqual instead.
  json.loads(json.dumps(val, bigint_as_string=True)))
....../tmp/simplejson-3.0.4/simplejson/tests/test_decode.py:17: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(rval, decimal.Decimal('1.1'))
./tmp/simplejson-3.0.4/simplejson/tests/test_decode.py:29: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(rval, {"key":"value", "k":"v"})
.../tmp/simplejson-3.0.4/simplejson/tests/test_decode.py:22: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(rval, 1.0)
...../tmp/simplejson-3.0.4/simplejson/tests/test_default.py:9: DeprecationWarning: Please use assertEqual instead.
  json.dumps(repr(type)))
../tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:18: DeprecationWarning: Please use assertTrue instead.
  self.assert_(json.loads(json.dumps(c)) is c)
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:19: DeprecationWarning: Please use assertTrue instead.
  self.assert_(json.loads(json.dumps([c]))[0] is c)
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:20: DeprecationWarning: Please use assertTrue instead.
  self.assert_(json.loads(json.dumps({'a': c}))['a'] is c)
./tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:14: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(sio.getvalue(), '{}')
./tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:54: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps({}), '{}')
./tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:59: DeprecationWarning: Please use assertEqual instead.
  '{"false": true, "true": false}')
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:68: DeprecationWarning: Please use assertEqual instead.
  '{"2": 3.0, "4.0": 5, "6": true, "7": 0, "false": 1}')
.../tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:33: DeprecationWarning: Please use assertEqual instead.
  {expect: expect})
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:36: DeprecationWarning: Please use assertEqual instead.
  {expect: expect})
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:45: DeprecationWarning: Please use assertEqual instead.
  v1)
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:48: DeprecationWarning: Please use assertEqual instead.
  {'': v1})
/tmp/simplejson-3.0.4/simplejson/tests/test_dump.py:51: DeprecationWarning: Please use assertEqual instead.
  [v1])
../tmp/simplejson-3.0.4/simplejson/tests/test_encode_basestring_ascii.py:42: DeprecationWarning: Please use assertEqual instead.
  '%r != %r for %s(%r)' % (result, expect, fname, input_string))
...../tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:20: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.lineno, 2)
/tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:21: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.colno, 1)
/tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:22: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.endlineno, 3)
/tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:23: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.endcolno, 2)
./tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:34: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.lineno, 1)
/tmp/simplejson-3.0.4/simplejson/tests/test_errors.py:35: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(err.colno, 9)
../tmp/simplejson-3.0.4/simplejson/tests/test_fail.py:112: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(e.pos, 1)
/tmp/simplejson-3.0.4/simplejson/tests/test_fail.py:113: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(e.lineno, 1)
/tmp/simplejson-3.0.4/simplejson/tests/test_fail.py:114: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(e.colno, 1)
../tmp/simplejson-3.0.4/simplejson/tests/test_float.py:10: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(json.dumps(inf)), inf)
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:13: DeprecationWarning: Please use assertTrue instead.
  self.assert_((0 + nan) != nan)
./tmp/simplejson-3.0.4/simplejson/tests/test_float.py:18: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(float(json.dumps(num)), num)
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:19: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(json.dumps(num)), num)
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:20: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(text_type(json.dumps(num))), num)
./tmp/simplejson-3.0.4/simplejson/tests/test_float.py:24: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(num), str(num))
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:25: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(int(json.dumps(num)), num)
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:26: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(json.dumps(num)), num)
/tmp/simplejson-3.0.4/simplejson/tests/test_float.py:27: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(text_type(json.dumps(num))), num)
./tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:45: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h1, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:46: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h2, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:47: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h3, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:48: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h4, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:49: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(d3, expect.replace('\t', '  '))
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:50: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(d4, expect.replace('\t', '  '))
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:54: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(d2, expect)
./tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:60: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(d1, expected)
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:64: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(sio.getvalue(), expected)
./tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:78: DeprecationWarning: Please use assertEqual instead.
  json.dumps(lst, indent=0, separators=(', ', ': ')))
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:82: DeprecationWarning: Please use assertEqual instead.
  json.dumps(lst, indent=0, separators=(',', ': ')))
/tmp/simplejson-3.0.4/simplejson/tests/test_indent.py:86: DeprecationWarning: Please use assertEqual instead.
  json.dumps(lst, indent=0))
./tmp/simplejson-3.0.4/simplejson/tests/test_pass1.py:70: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(res, json.loads(out))
./tmp/simplejson-3.0.4/simplejson/tests/test_pass2.py:14: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(res, json.loads(out))
./tmp/simplejson-3.0.4/simplejson/tests/test_pass3.py:20: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(res, json.loads(out))
./tmp/simplejson-3.0.4/simplejson/tests/test_recursion.py:60: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(enc.encode(JSONTestObject), '"JSONTestObject"')
....../tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:28: DeprecationWarning: Please use assertEqual instead.
  (u'z\U0001d120x', 16))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:37: DeprecationWarning: Please use assertEqual instead.
  (u'z\U0001d120x', 5))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:41: DeprecationWarning: Please use assertEqual instead.
  (u'{', 8))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:45: DeprecationWarning: Please use assertEqual instead.
  (u'A JSON payload should be an object or array, not a string.', 60))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:49: DeprecationWarning: Please use assertEqual instead.
  (u'Unclosed array', 17))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:53: DeprecationWarning: Please use assertEqual instead.
  (u'extra comma', 14))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:57: DeprecationWarning: Please use assertEqual instead.
  (u'double extra comma', 21))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:61: DeprecationWarning: Please use assertEqual instead.
  (u'Comma after the close', 24))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:65: DeprecationWarning: Please use assertEqual instead.
  (u'Extra close', 14))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:69: DeprecationWarning: Please use assertEqual instead.
  (u'Extra comma', 14))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:73: DeprecationWarning: Please use assertEqual instead.
  (u'Extra value after close', 26))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:77: DeprecationWarning: Please use assertEqual instead.
  (u'Illegal expression', 21))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:81: DeprecationWarning: Please use assertEqual instead.
  (u'Illegal invocation', 21))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:85: DeprecationWarning: Please use assertEqual instead.
  (u'Numbers cannot have leading zeroes', 37))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:89: DeprecationWarning: Please use assertEqual instead.
  (u'Numbers cannot be hex', 24))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:93: DeprecationWarning: Please use assertEqual instead.
  (u'Too deep', 30))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:97: DeprecationWarning: Please use assertEqual instead.
  (u'Missing colon', 16))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:101: DeprecationWarning: Please use assertEqual instead.
  (u'Double colon', 15))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:105: DeprecationWarning: Please use assertEqual instead.
  (u'Comma instead of colon', 25))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:109: DeprecationWarning: Please use assertEqual instead.
  (u'Colon instead of comma', 25))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:113: DeprecationWarning: Please use assertEqual instead.
  (u'Bad value', 12))
/tmp/simplejson-3.0.4/simplejson/tests/test_scanstring.py:118: DeprecationWarning: Please use assertEqual instead.
  (c, 2))
./tmp/simplejson-3.0.4/simplejson/tests/test_separators.py:40: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h1, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_separators.py:41: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(h2, h)
/tmp/simplejson-3.0.4/simplejson/tests/test_separators.py:42: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(d2, expect)
.../tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:50: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads('"' + u + '"'), u)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:51: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads('"z\\ud834\\udd20x"'), u)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:45: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(u), '"\\ud834\\udd20"')
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:46: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(u, ensure_ascii=False), u'"\U0001d120"')
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:78: DeprecationWarning: Please use assertEqual instead.
  {'a': u'\xe9'})
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:14: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(ju, js)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:21: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(ju, js)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:26: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(j, '"\\u03b1\\u03a9"')
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:31: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(j, '["\\u03b1\\u03a9"]')
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:36: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(j, u'"' + u + u'"')
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:41: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(j, u'["' + u + u'"]')
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:98: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(doc1), doc_ascii)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:99: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(doc2), doc_ascii)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:100: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(doc1, ensure_ascii=False), doc_unicode)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:101: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(doc2, ensure_ascii=False), doc_unicode)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:87: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.dumps([], ensure_ascii=False)), text_type)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:88: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.dumps(0, ensure_ascii=False)), text_type)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:89: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.dumps({}, ensure_ascii=False)), text_type)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:90: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.dumps("", ensure_ascii=False)), text_type)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:108: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(s1), expect)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:109: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(s2), expect)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:110: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(s1, ensure_ascii=False), expect)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:111: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.dumps(s2, ensure_ascii=False), expect)
.../tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:58: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(json.loads(s), u)
./tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:81: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.loads(u'""')), text_type)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:82: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.loads(u'"a"')), text_type)
/tmp/simplejson-3.0.4/simplejson/tests/test_unicode.py:83: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(type(json.loads(u'["a"]')[0]), text_type)
./tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:20: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(res, json.load(sio, **kw))
/tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:29: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(self.loads(s, parse_float=Decimal), Decimal(s))
../tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:14: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(res, sio.getvalue())
/tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:25: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(self.dumps(d, use_decimal=True), str(d))
./tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:47: DeprecationWarning: Please use assertEqual instead.
  v)
../tmp/simplejson-3.0.4/simplejson/tests/test_decimal.py:37: DeprecationWarning: Please use assertEqual instead.
  {str(d): d})
.............s
----------------------------------------------------------------------
Ran 92 tests in 3.973s

OK (skipped=1)

Error installing on Python 3.2

I am getting the following error while installing through PIP:

C:\Python32\Scripts>pip install https://github.com/simplejson/simplejson/tarball
/master
Downloading/unpacking https://github.com/simplejson/simplejson/tarball/master
Downloading master (51Kb): 51Kb downloaded
Running setup.py egg_info for package from https://github.com/simplejson/simpl
ejson/tarball/master
Traceback (most recent call last):
File "", line 14, in
File "c:\users\akhil\appdata\local\temp\pip-9ad10l-build\setup.py", line 3
9
except DistutilsPlatformError, x:
^
SyntaxError: invalid syntax
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 14, in

File "c:\users\akhil\appdata\local\temp\pip-9ad10l-build\setup.py", line 39

except DistutilsPlatformError, x:

                             ^

SyntaxError: invalid syntax

Command python setup.py egg_info failed with error code 1 in c:\users\akhil\appd
ata\local\temp\pip-9ad10l-build
Storing complete log in C:\Users\Akhil\AppData\Roaming\pip\pip.log

I am running Python version 3.2. Please help me to solve this issue.

docs submodule uses a private path

The docs submodule uses a private [email protected] URL/path, which requires users to be authenticated (with a public key) when trying to check it out.

Provide more information in error messages

http://bugs.python.org/issue16009

Make json.tool work again on Python 3 and add tests

http://bugs.python.org/issue16549

Large number of nested objects triggers recursion error

Large nested JSON objects will raise a recursion error like this:

Traceback (most recent call last):
  File "stack.py", line 25, in <module>
    main()
  File "stack.py", line 15, in main
    dd = simplejson.loads(s)
  File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/__init__.py", line 413, in loads
    return _default_decoder.decode(s)
  File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/decoder.py", line 402, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/decoder.py", line 418, in raw_decode
    obj, end = self.scan_once(s, idx)
RuntimeError: maximum recursion depth exceeded while decoding a JSON document

Here is a script to reproduce:

import json
import simplejson

def main():
    s = []
    num = 900
    for i in range(num):
        s.append('{"foo":')
    s.append('1')
    for i in range(num):
        s.append('}')
    s = ''.join(s)
    try:
        dd = simplejson.loads(s)
    except Exception, exc:
        print 'simplejson', str(exc) 
    try:
        dd = json.loads(s)
    except Exception, exc:
        print 'json', str(exc)

if __name__ == '__main__':
    main()

The stdlib json module fails at about 400 nested objects whereas simplejson fails at about 1000 nested objects. I know it sounds excessive but is there a way to make this not be recursive? The code we use to validate Firefox Add-ons was hitting this limit in production because of complex nested structureds (maybe locale based, I forget exactly).

Speedups broken in Python 2.5

Compiling simplejson-2.1.3 with Python 2.5.2 gives a warning:

simplejson/_speedups.c: In function 'encoder_listencode_dict':
simplejson/_speedups.c:2279: warning: implicit declaration of function 'Py_SIZE'

and continues installing. Then simplejson silently goes very slowly, because "import simplejson._speedups" gives:

ImportError: /[...]/simplejson/_speedups.so: undefined symbol: Py_SIZE

Looks like Py_SIZE was added in Python 2.6. Commit db980b6 added a call to it.

simplejson 2.3.1: simplejson.tests.test_namedtuple.TestNamedTuple.test_namedtuple_dumps() fails when simplejson/_speedups.so extension module is present

If _speedups.so extension has been built, then test_namedtuple_dumps fails and a segmentation fault can also occur. This regression has been introduced in simplejson 2.3.1.

$ python2.7 setup.py build_ext -i
...
$ python2.7 simplejson/tests/__init__.py
..........................................................................F...................................................................................
======================================================================
FAIL: test_namedtuple_dumps (simplejson.tests.test_namedtuple.TestNamedTuple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/simplejson-2.3.1/simplejson/tests/test_namedtuple.py", line 54, in test_namedtuple_dumps
    self.assertEqual(d, json.loads(json.dumps(v)))
AssertionError: OrderedDict([('value', 1)]) != [1]

----------------------------------------------------------------------
Ran 158 tests in 2.785s

FAILED (failures=1)
$ python2.7 simplejson/tests/__init__.py
..........................................................................F........................................................................Segmentation fault

Deletion of *.pyc files avoids segmentation fault.

Fails where embedded copy in python 2.6 works

Running Django's test suite with simplejson 2.2.0 installed results in two Django test suite failures. If I relaunch it without simplejson installed, it works fine because the copy embedded with python 2.6 works.

======================================================================
ERROR: test_serialize_unicode (modeltests.serializers.tests.JsonSerializerTestCase)
Tests that unicode makes the roundtrip intact
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/tests/modeltests/serializers/tests.py", line 172, in test_serialize_unicode
    obj_list = list(serializers.deserialize(self.serializer_name, serial_str))
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/core/serializers/json.py", line 35, in Deserializer
    for obj in PythonDeserializer(simplejson.load(stream), **options):
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/core/serializers/python.py", line 128, in Deserializer
    data[field.name] = field.to_python(field_value)
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/db/models/fields/__init__.py", line 761, in to_python
    return decimal.Decimal(value)
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string

======================================================================
ERROR: test_json_serializer (regressiontests.serializers_regress.tests.SerializerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/utils/functional.py", line 55, in _curried
    return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/tests/regressiontests/serializers_regress/tests.py", line 373, in serializerTest
    for obj in serializers.deserialize(format, serialized_data):
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/core/serializers/json.py", line 35, in Deserializer
    for obj in PythonDeserializer(simplejson.load(stream), **options):
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/core/serializers/python.py", line 128, in Deserializer
    data[field.name] = field.to_python(field_value)
  File "/home/rhertzog/deb/pkg/TEAMS/build-area/python-django-1.3.1/django/db/models/fields/__init__.py", line 761, in to_python
    return decimal.Decimal(value)
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string

No hooks to intercept encoding early.

The current API allows for extending encoding to more complex types by overriding the default method. However this happens last, after encoding for base types. But if you have an object that is an instance of a subclass of a base type the default method won't be called, and the object is encoded as the base type. Thus you lose the real type information. For example, specialiazed dictionaries that subclass dict get encoded as dict, but lose the superclass status, and can't override it to add custom metadata.

Use (",", ": ") as default separator when indent is specified to avoid trailing whitespace

http://bugs.python.org/16333

Pre-process Hooks

It would be great, if there was a way to "override" or some "pre_process" hook to execute, before default is fired. Currently there is no way this can be acheieved. and pre processing the data before feeding it to simple json, makes it a 2 x Loop.

Basically, Its a collection of objects which need to Inflated using one dynamic variable. The collection is sort of cached, and is kin of large in number.

In the comment update you have mentioned, it will be too overhead to implement.

Is there any other workaround you suggest [ i might have missed in the documentation ]?
I really wish to avoid that 1 extra iteration.

setup.py fails with 2.1.4

LONG_DESCRIPTION = open('README.rst', 'r').read()

IOError: [Errno 2] No such file or directory: 'README.rst'

Bad error on unclosed quote and brace

This is a bit of a weird error:

>>> simplejson.loads('{"asdf" : "')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jason/src/apture/lib/python2.7/site-packages/simplejson/__init__.py", line 385, in loads
    return _default_decoder.decode(s)
  File "/home/jason/src/apture/lib/python2.7/site-packages/simplejson/decoder.py", line 402, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/jason/src/apture/lib/python2.7/site-packages/simplejson/decoder.py", line 418, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: end is out of bounds

This should probably be JSONDecodeError.

Default direttive ignored

I would like to override the serialization of NaN fields. Consider for example this piece of code:

import simplejson

def encode_nan(obj):
if isinstance(obj, NaN):
return 0
raise TypeError(repr(obj) + " is not JSON serializable")

print simplejson.dumps(float('nan'), default=encode_nan)

The output in this case is

NaN

instead as far as I've unserstood from the documentation the output should be

Using simplejson 2.2.1

Does it make sense to make Python 3 dict_keys serializable?

I ran into an issue porting a project to Python 3, where simplejson raises a TypeError because of dict_keys:

  File "/Users/marca/dev/git-repos/cornice/.tox/py33/lib/python3.3/site-packages/simplejson/encoder.py", line 226, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: dict_keys([1, 2]) is not JSON serializable

This is easily fixed in the project by wrapping the dict_keys with list and that's fine, but I wonder if it makes sense to do it in simplejson as a convenience to make porting to Python 3 easier.

I could submit a PR if this is deemed useful, but I wanted to check first and make sure that this makes sense, wouldn't hurt performance too much, etc.

no JSONEncoderForHTML attribute in simplejson

import simplejson
simplejson.JSONEncoderForHTML()
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'JSONEncoderForHTML'

Escaping Issue when using ensure_ascii=False

[deals] > python
Python 2.7.2+ (default, Jul 20 2012, 22:12:53) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson
>>> simplejson.__version__
'3.0.5'
>>> print simplejson.dumps('a\\b', ensure_ascii=False)
"a\b"

(This could probably break a lot of software. I noticed that the Django Tastypie library is broken.)

json decoder question

How can I handle such a case with simplejson?

import simplejson
s = '[{"type":"image","width":100,"height":100,"x":63,"y":169,"src":"http://localhost:8000/media/images/anna.jpg","transform":""},{"type":"image","width":100,"height":100,"x":336,"y":163,"src":"http://localhost:8000/media/images/anna1.jpg","transform":""},{"type":"text","font":"10px \"Arial\"","font-size":20,"stroke":"none","fill":"#000","x":204,"y":90,"text":"default text","text-anchor":"middle"}]'
simplejson.loads(s)

JSONDecodeError Traceback (most recent call last)
/home/alexk/prj/94x54/ in ()
----> 1 simplejson.loads(s)

/home/alexk/.virtualenvs/94x54/local/lib/python2.7/site-packages/simplejson/init.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
411 parse_constant is None and object_pairs_hook is None
412 and not use_decimal and not kw):
--> 413 return _default_decoder.decode(s)
414 if cls is None:
415 cls = JSONDecoder

/home/alexk/.virtualenvs/94x54/local/lib/python2.7/site-packages/simplejson/decoder.pyc in decode(self, s, _w)
400
401 """
--> 402 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
403 end = _w(s, end).end()
404 if end != len(s):

/home/alexk/.virtualenvs/94x54/local/lib/python2.7/site-packages/simplejson/decoder.pyc in raw_decode(self, s, idx)
416 """
417 try:
--> 418 obj, end = self.scan_once(s, idx)
419 except StopIteration:
420 raise JSONDecodeError("No JSON object could be decoded", s, idx)

JSONDecodeError: Expecting , delimiter: line 1 column 296 (char 296)

Simplejson 2.3.0 encode issue with overridden dict getattr

Hi There,
Thanks very much for the hard work on simplejson. Evite uses json heavily, and the c speedups are amazing.

Anyhoo, since upgrading to 2.3.0, our object-like dict will no longer encode. Please see the following gist for a full example:

https://gist.github.com/1534953

I am not sure this is something that you will care to support, but I at least wanted to mention it. Thanks again.

simplejson should threat iterable as list

Is bad joke to have to do list(myiterableobject) to be able to serialize list-like object.

Inline already serialized JSON

Hello,
I'm currently working on a JSON API for my project. There are chunks of response that could be cached. So, I thought that it would be nice to have something like this:

>>> JSONEncoder().encode({'a': 10, 'b':  AlreadyJSON('{"key": "value"}')})
'{"a": 10, "b": {"key": "value"}}'

Sure, for such a simple case strings concatenation would do, but for cases with more complex JSON structure such an API would be handy.

I looked through the documentation and have not found any way to get this behavior.
So I wonder if there is a way to do it without modifying simplejson source code.

Sorry, if it is not an appropriate place for asking such questions.