reagento / adaptix Goto Github PK

Do serialization to primitives, that can be stored in extrernal structured format (e.g. json) and than parsed

Trying to deserialize a nullable Dict[T, U] results in get_dict_parser being called anyway and raising an AttributeError due to the target dict being None.

Testing with Python 3.7 and dataclass_factory 2.4.1 the following code will raise:

from typing import Optional, Dict, Any
from dataclass_factory import Factory

@dataclass
class Test:
    x: Optional[Dict[str, Any]]

f = Factory()
f.load({"x": None}, Test)

The root issue seems to be that Optional is not being detected correctly as it is indistinguishable from Union[T, NoneType], which ends up in get_dict_parser raising due to the order of the type arguments.

https://github.com/Tishka17/dataclass_factory#schemas
from dataclass_factory import Schema
skipped in example

It is very difficult to understand what going wrong during parsing

There is no Schema parametr, which do not change fields namestyle to one concrete, like bool ignore_namestyle

A lot of parameters, which description we have at docs, have no practical examples, which can helps to understand it better. Could you write more examples in the documentation?

I have a dataclass which str item default is "a" and int item default is 2.

But in data dict, their value is None.

When I call factory.load(data, DataClass), I want to leave the None type as it is, don't change it.
Or I don't want to get an exception when type .

Any solution?
Thanks

Store all extra input fields, which absent in dataclass, separately in field with name from schema.rest, if provided.

Also unpack rest when serialising

1. Support of creating TypedDict instances (from both mypy_extensions and typing) from normal Dict.
2. Do not break compatibility with 3.6
3. Do not add new dependencies
4. Need validation of __total__

In werkzeug MultiDict allows to use getlist to get multiple values for single name in form data.

This could be useful for filing dataclass List field

Custom parser can be provided in Schema.

It is ok to parse data in dict before passing to custom parser or provide a way to call use factory inside custo mparser.

1. Pass factory instance
2. Check custom parser anotations
3. Add zero-step parsing setting (e.g. link to other type)

@dataclass 
class List:
   data: int
   next: Optional[List]

This should be parsed as well

• name styles
• dataclass
• list
• dict
• tuple
• Any
• Optional
• Union
• classes
• cache serializers
• custom serializers

Tuple type; Tuple[X, Y] is the type of a tuple of two items with the first item of type X and the second of type Y.

Example: Tuple[T1, T2] is a tuple of two elements corresponding to type variables T1 and T2. Tuple[int, float, str] is a tuple of an int, a float and a string.

To specify a variable-length tuple of homogeneous type, use literal ellipsis, e.g. Tuple[int, ...]. A plain Tuple is equivalent to Tuple[Any, ...], and in turn to tuple.

see feature/schema_serial

Make JSONEncoder compatible version of factory

https://docs.python.org/3/library/json.html#json.JSONEncoder

We can parse class if it has init with annotations. Probably it is ok to serialize it as well. All schema features must be included

@dataclass
class A:
    x: str
    y: str


schema_num = Schema[A](
    name_mapping={
        "x": (1, "a", "b"),
    }
)


class Test1(TestCase):
    def test_load_num(self):
        factory = Factory(
            schemas={
                A: schema_num
            }
        )
        data = {
            "a": [None, {"b": "hello"}],
            "y": "world"
        }
        expected = A("hello", "world")
        self.assertEqual(expected, factory.load(data, A))

Ошибка:

======================================================================
ERROR: test_load_num (test_path.Test1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tishka17/src/dataclass_factory/tests/test_path.py", line 39, in test_load_num
    self.assertEqual(expected, factory.load(data, A))
  File "/home/tishka17/src/dataclass_factory/dataclass_factory/factory.py", line 94, in load
    return self.parser(class_)(data)
  File "/home/tishka17/src/dataclass_factory/dataclass_factory/parsers.py", line 139, in dataclass_parser
    for field, name, parser in field_info
TypeError: __init__() missing 1 required positional argument: 'x'

https://github.com/Tishka17/dataclass_factory#polymorphic-parsing
Missing import dataclass

Create instruction or provide builin mechanism for creating polymorphics parsers.

Idea: check class __init__ method args

from dataclasses import dataclass
from dataclass_factory import Factory
from datetime import datetime


@dataclass()
class Traffic:
    time: datetime
    rx: int
    tx: int


if __name__ == '__main__':
    dict_traff = {
        'time': datetime(2019, 12, 1, 1, 20, 13),
        'rx': 10,
        'tx': 15
    }

    factory = Factory()
    traff: Traffic = factory.load(dict_traff, Traffic)
    print(factory.dump(traff))
    
    

Ошибка:

Traceback (most recent call last):
  File "E:/NMS/NMSProgect/modules/traffic.py", line 24, in <module>
    traff: Traffic = factory.load(dict_traff, Traffic)
  File "E:\NMS\lib\site-packages\dataclass_factory\factory.py", line 94, in load
    return self.parser(class_)(data)
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 139, in dataclass_parser
    for field, name, parser in field_info
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 140, in <dictcomp>
    if name in data
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 186, in class_parser
    k: parser(data.get(k)) for k, parser in parsers.items() if k in data
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 186, in <dictcomp>
    k: parser(data.get(k)) for k, parser in parsers.items() if k in data
TypeError: argument of type 'datetime.datetime' is not iterable

Do you think to write the example in the docs with many parameter (like marshmallow), i.e. list comprehesion or as an alternative with a new dataclass and List[Item]?

What will happen if some of the fields are in one style and some in another? How does it work in practice? I would like to see more examples in Readme.nd

Пример:

T = TypeVar('T')

@dataclass
class Foo(Generic[T]):
    value: T

Должен парситься Foo[int]
Он не детектится как датакласс, надо работать с Origin

node.js возвращает поля с спецсимволами "foo-bar$1"
былобы удобно обьявить поле value_1 а получать значения из value-1

• check value
• check type
• support multiple values
• both typing in python 3.8 and typing_extensions

Example with https://github.com/Tishka17/dataclass_factory#structure-flattening full of magic
Could you explain how this works? So far, all this is not obvious. What kind of magic zero..?

Допустим есть классы с наследованием

@dataclass
class A:
   a: int

@dataclass
class B(A):
   b: int

Сейчас, если мы хотим сделать предобработку поля a, мы должны сделать две схемы: для класс A и для класса B и в обеих определить pre_parse.

Необходимо продумать как это делать более удобно

When serialising skip None values if it is also set as default

We need new field in Schema and change logic of dataclass serializer.

Skipping None for all Optionals is not safe, as object cannot be created from such a dict.

Skipping other defaults is not safe enough, but can be separatly implemented

Setting this feature for each field should be done separately

аналогичная функциональность в Pydantic

model_a.py

from pydantic import BaseModel


class ModelA(BaseModel):
    field: int

model_b.py

from __future__ import annotations

from typing import TYPE_CHECKING

from pydantic import BaseModel

if TYPE_CHECKING:
    from model_a import ModelA


class ModelB(BaseModel):
    field: int
    a: ModelA

main.py

from model_b import ModelB

print(ModelB.__fields__)
-> {'field': ModelField(name='field', type=int, required=True), 'a': ModelField(name='a', type=ForwardRef('ModelA'), required=PydanticUndefined)}

по type=ForwardRef и в этом месте неизвестно что именно из себя представляет ModelA для ModelB потому что его нет в скоупе модуля model_b.

для этого есть ModelB.update_forward_refs()
вызываем из main.py
только прямой вызов ModelB.update_forward_refs() ничего не сделает - NameError: name 'ModelA' is not defined, потому что в модуле где находится класс все еще нет такого имени

делаем следующее - ModelB.update_forward_refs(ModelA=ModelA)

и далее снова смотрим поля:
{'field': ModelField(name='field', type=int, required=True), 'a': ModelField(name='a', type=ModelA, required=True)}
ModelA теперь имеет реальный тип а не референс

Нужно поднять версию Programming Language так как с 3.7 добавлены датаклассы
https://www.python.org/dev/peps/pep-0557/

How can I deserialize a recursive structure?

I have seen #17 but I am getting errors trying to deserialize the following:

@dataclass(frozen=True)
class JsonSport:
    sport_id: int
    sport_name: str
    sport_description: Optional[str]
    sports: Optional[List["JsonSport"]]

The error is
dataclass_factory.exceptions.InvalidFieldError: Invalid data at path [sports]: No suitable parsers in union found for [{'sportId': 63, 'sportName': 'Idrettsskoler', 'sportDescription': 'Idrett generelt...`

Did not see custom validation of fields except types. For example here https://marshmallow.readthedocs.io/en/stable/quickstart.html#validation . For example, expect a certain length or a certain value from a list of valid. Also multiple validation of one field, for example a password field

Hi! Thanks for the awesome project.

It looks like you have covered almost all of your source code with type annotations, but currently it is not shipped to the final user.

I suggest to fix this situation. Here's how:

1. Create py.typed file in the root of the dataclass_factory folder, it is required for PEP561 to work. Example: https://github.com/dry-python/returns/blob/master/returns/py.typed
2. Change how public API is typed. Some method do miss return values. Some of them are missing generic parameters. Like this one: https://github.com/Tishka17/dataclass_factory/blob/master/dataclass_factory/factory.py#L80 It should be something like: def load(self, data: Any, class_: Type[T]) -> T: (do not forget about Generic base class)
3. Integrate mypy to check these annotations

This feature would improve developer experience and make this project even more awesome!

Add to schema "impl" attribute which provides real type of interface.

So factory.load(data, Interface) will be equal to factory.load(data, impl)
Also must be applyed to fields with Interface type

dataclass <-> dataclass
dataclass <-> protobuf