The 'namedtuple_as_object' kwarg causes backwards incompatibility about simplejson HOT 8 CLOSED

I don't think I consider this to be a bug, I can't think of a good reason why this must work. If you can call simplejson you can subclass the encoder from there instead of the stdlib.

from simplejson.

It's not so much a bug as a feature request. Django doesn't want to include simplejson as a dependency, and so we can't subclass simplejson's encoder unless we do some hackery at load time to detect that simplejson exists and subclass it if and only if it does. It has been suggested that Django move away from this hackery because the standard library json package is available on all supported python versions and is capable of doing what we want.

The problem is that due to simplejson.dumps() calling __init__() with the namedtuple_as_object keyword argument, Django 1.5's encoder will be incompatible with simplejson.dumps().

A consensus seems to be shaping that we should just document that simplejson has diverged from the standard library, recommend that no one use it, and leave it at that.

from simplejson.

This import hackery is the only way to do it. Even if the constructor was compatible, it would have incorrect behavior. You'd be doing your users a disservice to recommend that they use the much slower and less flexible stdlib implementation in cpython 2.x.

from simplejson.

"If you can call simplejson you can subclass the encoder from there instead of the stdlib."
And if I can't because I'm not calling but one library in my stack is ?
Thanks..

from simplejson.

"You'd be doing your users a disservice to recommend that they use the much slower and less flexible stdlib implementation in cpython 2.x."

By being incompatible with all other JSON implementations, users may be pushed to stop using simplejson > 2.1

Django isn't one project. It's a core with a huge network of pluggable apps. If one app can use simplejson, but it's presence breaks other apps, then simplejson is the problem and the user must either remove it and use stdlib, or downgrade to 2.1.

from simplejson.

I don't consider the ability to subclass something from one library and use it in another to be a useful definition of compatibility. No other project has ever expressed a need for this, and I have never seen this kind of hack used in production code anywhere else in my career.

I'm sorry, but the right thing to do us to fix Django properly.

from simplejson.

For Django, the choice is between having a soft dependency on simplejson for dubious benefit, and simply depending on the standard json package included in every Python standard library since 2.6. It seems obvious what the best choice is for their project.

As for the speedups, they were found to be nonexistent as of python 2.7 (see https://code.djangoproject.com/ticket/18023). These speedups are anyways dependent on having the C extensions compiled, which is easier to guarantee for the Python standard library than for third-party dependencies. For example, I deploy webapps to a server without a compiler, where Python and the database adapter I use come pre-built from the distro vendor, but simplejson has to be pulled from PyPI. If I used simplejson, I would definitely see a slowdown.

As for improvements in simplejson, while there are some nice new features since 2.0.9, there are also some bugs that were fixed in Python's standard json library that are still present in simplejson. The biggest one that stands out to me is that, contrary to its documentation, simplejson returns str instances instead of unicode strings if and only if the C extensions were compiled and the string is all ASCII. For a web framework like Django that has converted wholesale to unicode strings everywhere, that is broken behavior.

So I don't think you are going to convince Django to go back to simplejson (they already released 1.5 without it). Instead, unless you regain backwards compatibility with the json package from Python's standard library, you will continue to force people to make a choice between simplejson and the Django serialization framework, and I think a lot of people will choose the latter.

from simplejson.

There are no known bugs present in current simplejson that have been fixed in the stdlib. I'd be very happy to fix any of them if they were to be reported.

The speed/memory efficiency of using 8 bit ASCII str when possible was VERY significant for us at the time, when working with very large JSON documents. You can easily get around this by ensuring that the input is unicode rather than str, which is fully compatible with all versions of json and simplejson. This is documented behavior in load/loads, although admittedly this documentation was written after the implementation. You'll be happy to know that this hack is not implemented in Python 3 due to PEP 393 which gives you all of the benefits of a compact representation with none of the downside of using two types for strings.

I'm unwilling to stop adding new kwargs as appropriate or add hacks to support using subclasses of json classes with simplejson's codecs. simplejson's behavior isn't broken, the expectations and implementation of these subclasses in Django is. I'd be happy to support a reasonable compromise, but beyond that it's not really my concern what Django decides to do. It's not even necessary to use subclassing as all of the useful functionality can be done with arguments to loads/dumps. Specifically, the function args object_hook (for loads) and default (for dumps) give you pretty much all of the power you need to parameterize behavior without depending on any implementation details.

from simplejson.