use __json__ attribute to encode custom objects about simplejson HOT 13 CLOSED

I'd like to see this feature implemented. I've been working with different projects that each override JSONEncoder.default in order to encode their novel objects. It's generally unsustainable for each project that needs to customize json encoding to have to have access to the encoding step. It would be preferable for a hook like this to generalize custom encoding. Extra points for a corollary hook in the decode step. Consider this:

class MyCustom(object):
    def __json__(self):
        return {
            'a': self.a,
            'b', self.b,
            '__python__': {'mymodule:MyCustom.from_json'},
        }

    @classmethod
    def from_json(cls, json):
        obj = cls()
        obj.a = json['a']
        obj.b = json['b']
        return obj


import simplejson

obj = MyCustom()
obj.a = 3
obj.b = 4

json = simplejson.dumps(obj)
obj2 = simplejson.loads(json)
assert isinstance(obj2, MyCustom)
assert obj2.__dict__ == obj.__dict__

The __json__ method should return a Python dictionary (or other json-encodable primitive) suitable for encoding in json. In the decoding step, if the decoder finds an attribute __python__ (or similar), it will resolve that string to a callable (using the '{module path}:{callable}' convention), and then call that callable with the decoded json.

This protocol would enable a wide array of objects to be encoded and decoded using simplejson without any alteration of the calls to dumps/loads.

from simplejson.

Not very scientific, but, searching github restricted to repositories tagged as python yields:

__json__ - 4,034 instances
for_json - 3,856 instances

from simplejson.

Special methods starting and ending with double underscores are reserved for Python interpreter. If once the standard json module would support special attribute to encode custom objects, it would likely be named json. But it's signature can be different.

from simplejson.

There's a "for_json" hook that does what you need on encoding, added in 3.2.0.

class MyObj(object):
    def for_json(self):
        return {'a': self.a, 'b':self.b}

simplejson.dumps(MyObj(), for_json=True)

You can implement a simple wrapper to call your from_json on decoding without having it built in to loads, where if it finds a key you specify as a loading function in the JSON, it will process the JSON with it.

Though, having that string exposed to any user-input is a large security risk. And if you're not exposing it to users (e.g. saving to disk, database), the Pickle/cPickle library is probably a better approach, and does what you want already.

from simplejson.

Where is this 'repr(o)' thing coming from? The default behavior of the encoder is to raise an exception for instances it does not know how to encode. Are you really sure that both libraries are using simplejson? What is this thing other than Pyramid that you are using? Why can't your library just deal in data structures of serializable types, rather than instances of a custom class? How does decoding work?

The more detail you provide, the better. I'm not sure that any change to simplejson will really help you, or if it would, exactly what change should be made that doesn't break anyone else's code.

from simplejson.

My apologies, ignore the repr(o) part. I was mistaken because that is part of the Traceback that IPython gives me when I try and serialize my object with json.

The reason I'm using a custom object is because we've created a custom Money class to handle prices in our system. The Money class stores its value internally as a Decimal value, but I wasn't able to subclass Decimal due to restrictions in its design. So serializing should be a simple matter of returning the internal Decimal value as a string (or to return a Decimal and use the recent addition of Decimal handling in simplejson).

The other library I'm using is riak-python. So a dict is passed in to riak with (for instance) the price field set to a Money object. Riak then attempts to serialize this data and fails.

Any assistance you can provide would be most appreciated. I understand the need to limit any new features to a library focused on simplicity, but the current method to use custom objects by subclassing the JSONEncoder object and passing it to json.dumps only works if you also have control over everything that uses simplejson.

from simplejson.

I was just going through the code, and noticed that there is a similar code path for namedtuple that looks for a method called _asdict(). While this doesn't quite work for me, since I don't want to represent my data as a dict, it is similar to the behavior I'd like to see, especially since _asdict() doesn't check the type of the object via isinstance(). Would something similar (i.e. calling the str method like the code path for Decimal already does) be possible? I'm not sure I see the reasoning behind rejecting any custom object, even if it makes methods available to be parseable to outside code.

from simplejson.

Pull request here: #54

from simplejson.

So there's good news, and there's bad news. The good news is that you can replace the dumps implementation in riak-python-client rather easily, see https://github.com/basho/riak-python-client/blob/master/riak/client.py#L150 - you should be able to accomplish what you want with no changes to any library.

The bad news is that riak-python-client does the wrong thing and imports json first, then simplejson if json doesn't import (only true on Python 2.5 and older). This is bad because the json library is always slower than simplejson, and of course it's not going to benefit from any changes that you make to your simplejson installation. https://github.com/basho/riak-python-client/blob/master/riak/client.py#L22

from simplejson.

@etrepum @shakefu would you consider allowing the for_json parameter of dumps to accept a string of the name of the method in addition to a bool? if the value is True, it continues to do the current behavior of calling for_json on the object. If the value is a string, it calls the method by that name instead (such as dunder json, __json___).

from simplejson.

If there was a well tested PR and some significant examples of __json__ usage in the wild that made it compelling.

from simplejson.

If once the standard json module would support special attribute to encode custom objects, it would likely be named __json__. But it's signature can be different.

This point seems like an argument for using __json__, but with the additional caveat that the protocol should be presented in a PEP and accepted first, so that the signature can be formalized and tools like simplejson can rely on the same protocol.

from simplejson.

@jaraco It really isn't, since if the _json__ implementation differed at all from whats simplejson used (say, it didn't return a JSON-encodable object, but returned str or bytes, the actually encoded JSON), then simplejson would have to break backwards compatibility and break user code. If __json__ is ever used by the Python interpreter, and the two are identical, then simplejson can implement __json__ as for_json.

from simplejson.