ikornaselur / dict-typer Goto Github PK

View Code? Open in Web Editor NEW

18.0 18.0 0.0 1.47 MB

Convert JSON or Python dictionaries to Python TypedDict definitions

License: MIT License

Makefile 0.89% Python 98.91% Shell 0.20%

json mypy python

dict-typer's People

Contributors

Stargazers

Watchers

dict-typer's Issues

Handle different types between list of dicts

Minimal example:

[{"item": 123}, {"item": "foo"}]

should produce

class RootTypeItem(TypedDict):
    item: Union[int, str]

RootType = List[RootTypeItem]

but now creates

class RootItemType(TypedDict):
    item: int


class RootItemType1(TypedDict):
    item: str


RootType = List[Union[RootItemType, RootItemType1]]

Add imports automatically to the output

Add relevant imports from typing and typing_extensions

Support dicts in lists

Minimal example

{
  "dictList": [{ "id": 123 }, { "id": 456 }]
}

Support definitions with builtin key names

Switch to alternative definition of a TypedDict when using any keys with reserved keywords. Keywords can be queried from keyword:

In [2]: import keyword

In [3]: keyword.kwlist
Out[3]:
['False',
 'None',
 'True',
 'and',
 'as',
 'assert',
 'async',
 'await',
 'break',
 'class',
 'continue',
 'def',
 'del',
 'elif',
 'else',
 'except',
 'finally',
 'for',
 'from',
 'global',
 'if',
 'import',
 'in',
 'is',
 'lambda',
 'nonlocal',
 'not',
 'or',
 'pass',
 'raise',
 'return',
 'try',
 'while',
 'with',
 'yield']

Support lists in the root

At the moment, it's required that the root object is a dictionary.

Support root object being a list by either:

Just always work on the first item of the list (simpler I guess)
Going through the list and processing each item in the list to find dicts

If the input is a list in 2., then output a comment that lists the types of the input list]

Will likely be what will be needed in #11, for completeness, which can then be used here in the root

Example for 2.

Input

[
  1,
  2,
  {"id": 123},
  {"id": 456},
]

Output

from typing_extensions import TypedDict


class RootType(TypedDict):
    id: int

# Input: Union[int, RootType]

typing_extensions should be a dev depenency

While the output of this tool uses TypedDict from typing_extensions heavily, the actual core tool itself doesn't. It's only used in tests.

Review empty lines

Review empty lines between definitions and imports

Support detecting sequence type from other typed dicts if possible

Minimal example:

[{"items": []}, {"items": [1, 2, 3]}]

should produce

from typing import List

class RootTypeItem(TypedDict):
    items: List[int]

RootType = List[RootTypeItem]

but now creates

from typing import List, Union

from typing_extensions import TypedDict


class RootItemType(TypedDict):
    items: Union[List, List[int]]


RootType = List[RootItemType]

Add integration tests with json examples

Take the examples from https://json.org/example.html and use them as integration tests

Fix order of definitions

The root type has to come after any nested definitions

Introduce interactive CLI support

Add an interactive option to the CLI, which should:

Parse as normally then show the results
Show a list of the definition names (or a number shown with each in the result output?)
Allow user to finalise, which prints out the result
Allow the user to pick each definition by number and rename it

Fix missing imports for root list types

Check with [1, 2.0, "3"] and show imports to reproduce, which should import Union and List

Replace dataclass to support 3.6

Update the CLI experience

Update the CLI experience with the following things:

Integrate Click for --help
Support for defining input file

Support dataclasses + optionally `desert`?

https://pypi.org/project/desert/

Nested same keys throws error

 {"foo": {"foo": 10}}

will result in

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.8.2/bin/dict-typer", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/__init__.py", line 50, in cli
    output = get_type_definitions(parsed, show_imports=imports)
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/type_definitions.py", line 229, in get_type_definitions
    return builder.build_output()
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/type_definitions.py", line 180, in build_output
    typing_imports |= definition.get_imports()
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/models.py", line 132, in get_imports
    imports |= sub_member.get_imports()
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/models.py", line 132, in get_imports
    imports |= sub_member.get_imports()
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/models.py", line 132, in get_imports
    imports |= sub_member.get_imports()
  [Previous line repeated 987 more times]
  File "/usr/local/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/dict_typer/models.py", line 129, in get_imports
    imports = set()
RecursionError: maximum recursion depth exceeded while calling a Python object

Support custom data types

When an unknown data type is encountered, try to handle it:

If it's a class instance, track the instance.__class__.__name__ and list as a comment in the imports

Support extracting large definitions

Some definitions might be annoyingly long, such as

itemsTuple: Tuple[Union[Set[Union[float, int]], int]],

so support extracting this out in a separate alias, such as

ItemsTupleType = Tuple[Union[Set[Union[float, int]], int]]
...
itemsTuple: ItemsTupleType

Handle empty dicts

Minimal example:

{}

{"foo": {}}

Should either default to just Dict or a TypedDict with pass as body

Release on pypi

Detect repeated definitions in deeply nested dicts

Minimal example to reproduce

In [13]: d
Out[13]:
{'nest': {'a': 'a'},
 'deep_nest': {'deeply_nested': {'a': 'a'},
  'deep_deep_nest': {'nest': {'a': 'a'}}}}

In [14]: print(convert(d))
class NestType(TypedDict):
    a: str

class DeeplyNestedType(TypedDict):
    a: str

class NestType(TypedDict):
    a: str

class DeepDeepNestType(TypedDict):
    nest: NestType

class DeepNestType(TypedDict):
    deeply_nested: DeeplyNestedType
    deep_deep_nest: DeepDeepNestType

class RootType(TypedDict):
    nest: NestType
    deep_nest: DeepNestType

Support all base Python types

Current focus has been support what JSON needs, but there are sequences such as tuples and sets in python that need to be supported.

I'm unsure of if there's some place to find all builtin types that make sense (such as keyword.kwlist for keywords), but at least keep set and tuples as first class citizens.

Provide naming map

Optionally support a naming map, to rename certain types. Mostly useful for pytyper.dev

Input

{
  "foo": {"count": 1},
  "bar": {"count": 2}
}

Current output

class Foo(TypedDict):
    count: int


class Root(TypedDict):
    foo: Foo
    bar: Foo

With naming map

{
  "Foo": "Count"
}

class Count(TypedDict):
    count: int


class Root(TypedDict):
    foo: Count
    bar: Count

ikornaselur / dict-typer Goto Github PK

dict-typer's People

Contributors

Stargazers

Watchers

dict-typer's Issues

Minimal example

Example for 2.

Input

Output

Input

Current output

With naming map

Recommend Projects

Recommend Topics

Recommend Org