Code Monkey home page Code Monkey logo

yapic.json's People

Contributors

agijsberts avatar alexwilliams0712 avatar bmoscon avatar laky55555 avatar zozzz avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

bmoscon laky55555 alexwilliams0712

yapic.json's Issues

Remove boolean options, and replace with flags

  • dumps -> dumps(obj, options: int, ...)

    • ensure_ascii -> DUMP_ENSURE_ASCII
    • encode_datetime -> DUMP_DATETIME
    • options default: DUMP_DATETIME

  • loads -> loads(s, options: int, ...)

    • parse_date -> LOAD_DATETIME, LOAD_DATETIME_STRICT, (maybe LOAD_DATE, LOAD_TIME), option for large millisecond handling
    • options default: LOAD_DATETIME | LOAD_DATE | LOAD_TIME

macOS compatibility

Any chance this could be made to work on macOS? Since no wheel is provided on pypi for macos, it attempts to build from source, and generates quite a few warnings and errors:

Searching for yapic.json>=1.4.1
Reading https://pypi.org/simple/yapic.json/
Downloading https://files.pythonhosted.org/packages/bb/43/ee6ab394aac8e83695c1cdc8a14eb63a4d1cc5da2eac56efa807f2aa8ae7/yapic.json-1.4.1.tar.gz#sha256=8c29c8718478168520eea3473e46c9bcc52d235ffbc9c99755ff2df569fbbdb0
Best match: yapic.json 1.4.1
Processing yapic.json-1.4.1.tar.gz
Writing /var/folders/_r/28vq05fx4mqf0fr5fcd0tg_40000gn/T/easy_install-khog2di8/yapic.json-1.4.1/setup.cfg
Running yapic.json-1.4.1/setup.py -q bdist_egg --dist-dir /var/folders/_r/28vq05fx4mqf0fr5fcd0tg_40000gn/T/easy_install-khog2di8/yapic.json-1.4.1/egg-dist-tmp-gfczgeh3
In file included from src/json.cpp:4:
In file included from src/encoder.h:9:
src/buffer.h:66:4: warning: 'register' storage class specifier is deprecated and
      incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t current_usage = cursor - start;
                        ^~~~~~~~~
src/buffer.h:67:4: warning: 'register' storage class specifier is deprecated and
      incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t new_size = end - start;
                        ^~~~~~~~~
src/buffer.h:307:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Chunk* c = chunksBegin;
                        ^~~~~~~~~
In file included from src/json.cpp:4:
src/encoder.h:237:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT* out = buffer.cursor;
                        ^~~~~~~~~
src/encoder.h:238:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT maxchar = buffer.maxchar;
                        ^~~~~~~~~
src/encoder.h:328:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register long long value = PyLong_AsLongLongAndO...
                        ^~~~~~~~~
src/encoder.h:332:12: warning: implicit conversion of NULL constant to 'bool'
      [-Wnull-conversion]
                                return NULL;
                                ~~~~~~ ^~~~
                                       false
src/encoder.h:336:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register unsigned long long abs_value = value;
                        ^~~~~~~~~
src/encoder.h:343:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT *end_position = buffer.cursor + L...
                        ^~~~~~~~~
src/encoder.h:344:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT *saved_end_position = end_position;
                        ^~~~~~~~~
src/encoder.h:380:5: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                                register int size = builder.position();
                                ^~~~~~~~~
src/encoder.h:670:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t length = 0;
                        ^~~~~~~~~
src/encoder.h:723:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t length = PyList_GET_SIZE(obj);
                        ^~~~~~~~~
src/encoder.h:724:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t i = 0;
                        ^~~~~~~~~
src/encoder.h:749:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t length = PyTuple_GET_SIZE(obj);
                        ^~~~~~~~~
src/encoder.h:750:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t i = 0;
                        ^~~~~~~~~
src/encoder.h:781:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register Py_ssize_t length = 0;
                        ^~~~~~~~~
In file included from src/json.cpp:20:
src/decoder.h:93:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT maxchar = 127;
                        ^~~~~~~~~
src/decoder.h:180:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT maxchar = 127;
                        ^~~~~~~~~
src/decoder.h:181:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register CHOUT tmp = 0;
                        ^~~~~~~~~
src/decoder.h:204:4: warning: unused label 'error' [-Wunused-label]
                        error:
                        ^~~~~~
src/decoder.h:668:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register typename Trait::Int intValue = 0;
                        ^~~~~~~~~
src/decoder.h:818:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register PyObject* list = PyList_New(0);
                        ^~~~~~~~~
src/decoder.h:819:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register PyObject* item;
                        ^~~~~~~~~
src/decoder.h:869:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register PyObject* dict = PyDict_New();
                        ^~~~~~~~~
src/decoder.h:870:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register PyObject* key = NULL;
                        ^~~~~~~~~
src/decoder.h:871:4: warning: 'register' storage class specifier is deprecated
      and incompatible with C++17 [-Wdeprecated-register]
                        register PyObject* value = NULL;
                        ^~~~~~~~~
In file included from src/json.cpp:4:
In file included from src/encoder.h:9:
src/buffer.h:160:15: error: dependent using declaration resolved to type without
      'typename'
                using Base::MemoryBuffer;
                            ^
src/encoder.h:68:20: note: in instantiation of template class
      'Yapic::Json::FileBuffer<unsigned char, 16384>' requested here
                typedef typename BUFF::Char CHOUT;
                                 ^
src/json.cpp:92:5: note: in instantiation of template class
      'Yapic::Json::Encoder<Yapic::Json::FileBuffer<unsigned char, 16384> >'
      requested here
                                __encode_new_encoder(FileBuffer, Py_UCS1);
                                ^
src/json.cpp:12:50: note: expanded from macro '__encode_new_encoder'
        Encoder< B<T, YAPIC_JSON_ENCODER_BUFFER_SIZE> > encoder; \
                                                        ^
src/buffer.h:42:7: note: target of using declaration
class MemoryBuffer {
      ^
src/buffer.h:160:15: error: dependent using declaration resolved to type without
      'typename'
                using Base::MemoryBuffer;
                            ^
src/encoder.h:68:20: note: in instantiation of template class
      'Yapic::Json::FileBuffer<unsigned int, 16384>' requested here
                typedef typename BUFF::Char CHOUT;
                                 ^
src/json.cpp:97:5: note: in instantiation of template class
      'Yapic::Json::Encoder<Yapic::Json::FileBuffer<unsigned int, 16384> >'
      requested here
                                __encode_new_encoder(FileBuffer, Py_UCS4);
                                ^
src/json.cpp:12:50: note: expanded from macro '__encode_new_encoder'
        Encoder< B<T, YAPIC_JSON_ENCODER_BUFFER_SIZE> > encoder; \
                                                        ^
src/buffer.h:42:7: note: target of using declaration
class MemoryBuffer {
      ^
In file included from src/json.cpp:4:
src/encoder.h:332:12: warning: implicit conversion of NULL constant to 'bool'
      [-Wnull-conversion]
                                return NULL;
                                ~~~~~~ ^~~~
                                       false
src/encoder.h:107:12: note: in instantiation of member function
      'Yapic::Json::Encoder<Yapic::Json::MemoryBuffer<unsigned char, 16384>
      >::EncodeLong' requested here
                                return EncodeLong(obj);
                                       ^
src/json.cpp:57:29: note: in instantiation of member function
      'Yapic::Json::Encoder<Yapic::Json::MemoryBuffer<unsigned char, 16384>
      >::Encode' requested here
                                if (EXPECT_TRUE(encoder.Encode(obj))) {
                                                        ^
In file included from src/json.cpp:4:
src/encoder.h:332:12: warning: implicit conversion of NULL constant to 'bool'
      [-Wnull-conversion]
                                return NULL;
                                ~~~~~~ ^~~~
                                       false
src/encoder.h:107:12: note: in instantiation of member function
      'Yapic::Json::Encoder<Yapic::Json::MemoryBuffer<unsigned int, 16384>
      >::EncodeLong' requested here
                                return EncodeLong(obj);
                                       ^
src/json.cpp:62:29: note: in instantiation of member function
      'Yapic::Json::Encoder<Yapic::Json::MemoryBuffer<unsigned int, 16384>
      >::Encode' requested here
                                if (EXPECT_TRUE(encoder.Encode(obj))) {
                                                        ^
29 warnings and 2 errors generated.
error: Setup script exited with error: command 'gcc' failed with exit status 1

I'm confused with Windows 2019 Server 64bit

I'm confused what is going on.
Any ideas are very warm welcome.

My steps:

  1. Fresh installation of Windows 2019 Server 64bit

  2. Python v3.7.5 is installed in C:\Run\Python37
    and it is added to PATH environment variable.

  3. From command line I call pip install yapic.json
    Which installs yapic.json v1.6.3
    Everything looks good.

  4. Now I try to run very simple script with call python test_yapic.py
    Here is this script:
    from yapic import json

    This call produces an error. Here is an output:
    Traceback (most recent call last): File "test_yapic.py", line 2, in <module> from yapic import json File "C:\Run\Python37\lib\site-packages\yapic\json\__init__.py", line 1, in <module> from ._json import __version__ # noqa ImportError: DLL load failed: The specified module could not be found.

Any ideas?
Thank you in advance.

ValueError: microsecond must be in 0..999999

I ran the following code.
However, only d1 returned the following error.
Is this normal behavior?

from yapic import json  
import json as j  
d1 = json.loads('{"exec_date":"2020-05-22T18:05:36.8975412Z"}')  
d2 = j.loads('{"exec_date":"2020-05-22T18:05:36.8975412Z"}')  

Traceback (most recent call last):  
  File "/home/ec2-user/environment/test.py", line 4, in <module>  
    d1 = json.loads('{"exec_date":"2020-05-22T18:05:36.8975412Z"}')  
ValueError: microsecond must be in 0..999999

UTF-8 Decoding?

The Readme states:

UTF-8 encoding & decoding support

But when I try and load a simple bytestring it fails:

from yapic import json
json.loads(b'{"a": 1}')

TypeError  
Traceback (most recent call last)
<ipython-input-5-c9f0bd3ca33c> in <module>
----> 1 json.loads(b'{"a": 1}')

TypeError: argument 1 must be str, not bytes

whereas with CPython json, this works:

import json as j

j.loads(b'{"a": 1}')

Out: {'a': 1}

What am I missing? Is there a kwarg I need to pass to have it decode?

`decimal.InvalidOperation` when `0E-8` in JSON

Thank you for your quick fix of my last issue report #5 🤩

New issue regarding same kind of JSON content.

import decimal
# import json
from yapic import json

msg_str = '{"cp":0E-8}'
msg_dict = json.loads(msg_str, parse_float=decimal.Decimal)
print(msg_dict)

Standard JSON package works fine

$ python3 test.py
{'cp': Decimal('0E-8')}

Using Yapic-1.4.5, decimal.InvalidOperation is raised

$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    msg_dict = json.loads(msg_str, parse_float=decimal.Decimal)
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

Exception class compatibility with builtin `json` module

Hi,

The error exception class name in the builtin json module is JSONDecodeError, while the exception class in this module is JsonDecodeError. If you could provide an alias JSONDecodeError = JsonDecodeError in this module (probably in __init__.py) it would make it easier to migrate between the two json modules and reduce typing error.

bytearray support

I'm using yapic.json to decode data received from a socket. For performance reasons, I'd like to switch to using recv_into with bytearrays, however yapic.json doesn't support decoding bytearrays.

The change itself seems quite trivial, however I did not manager to compile it myself because it fails to find the yapic.core headers for some reason.

Millisecond fraction of seconds always interpreted as microseconds

When the length of the fraction of seconds is not exactly 6, the current implementation implicitly fills the remaining spaces with leadings zeros (.123 -> .000123), whereas trailing zeros would probably make more sense (.123 -> .123000). As a specific example, milliseconds in the following example become microseconds:

>>> from yapic import json
>>> json.loads('{"timestamp":"2021-04-07T20:23:44.123Z"}')
{'timestamp': datetime.datetime(2021, 4, 7, 20, 23, 44, 123, tzinfo=datetime.timezone.utc)}
>>> _['timestamp'].isoformat()
'2021-04-07T20:23:44.000123+00:00'

Yapic should reconize `0E-8` as a number

The UpBit crypto-exchange provides the following JSON message containing "cp":0E-8.

{"ty":"trade","cd":"BTC-SRN","tms":1589198102665,"td":"2020-05-11","ttm":"11:55:02","ttms":1589198102000,"tp":8.8E-7,"tv":760.07406626,"ab":"ASK","pcp":8.8E-7,"c":"EVEN","cp":0E-8,"sid":15891981020000000,"st":"SNAPSHOT"}

Yapic raises the exception:

yapic.json.JsonDecodeError("Unexpected character found when decoding 'dict', expected one of ',', '}' at position: 176.")

Segmentation fault due to negative ref count on broken json input

Description

An interrupted json of a dictionary causes a negative ref count and therefore a segmentation fault.

Example

from yapic import json
s = '{"foo":"bar","baz'
json.loads(s)

Output

One would expect exception yapic.json.JsonDecodeError: Unexpected end of data at position: 17. with no other side effects. Instead with normal python we get a segmentation fault. With python with debug symbols we get:

../Include/object.h:497: _Py_NegativeRefcount: Assertion failed: object has negative ref count
<object at 0x7f04a0c37ae0 is freed>
Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized
yapic.json.JsonDecodeError: Unexpected end of data at position: 17.

Current thread 0x00007f04a10c2740 (most recent call first):
  File "/home/user/dev/git/rtdata/yapic_segfault.py", line 3 in <module>
Aborted

So the exception is raised correctly, but hidden due to a superfluous reference decrement. GDB seems to indicate https://github.com/zozzz/yapic.json/blob/master/src/decoder.h#L956 as the culprit, which I guess would make sense.

Environment

Latest wheel from PyPi (1.7.0) and latest git (1.6.3, strangely enough) on linux.

enum.IntEnum is not JSON serializable

Migrating from the stdlib json library to yapic.json fails with enum.IntEnum objects.

The stdlib module correctly handles enum.IntEnum as integer but yapic.json fails with

yapic.json.JsonEncodeError: <IntEnum.first: 1> is not JSON serializable.

To reproduce:

class SomeIntEnum(enum.IntEnum):
    first = 1
>>> json.dumps(SomeIntEnum.first)
'1'
>>> yapic.json.dumps(SomeIntEnum.first)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
yapic.json.JsonEncodeError: <SomeIntEnum.first: 1> is not JSON serializable.
>>> isinstance(SomeIntEnum.first, int)
True

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.