quandyfactory / dicttoxml Goto Github PK

View Code? Open in Web Editor NEW

378.0 378.0 147.0 1.25 MB

Simple library to convert a Python dictionary or other native data type into a valid XML string.

License: GNU General Public License v2.0

Python 100.00%

dicttoxml's People

Contributors

Stargazers

Watchers

Forkers

arnabkd 0902horn jkahn bentasker niusmallnan swordqiu dwcramer praju skatila leancz zrd vad gauravub murielsilveira lichenbo mfriedenhagen ckj634167 cerivera daniell aptivate isaac-councill ryancurrah jacobsvante jakecoppinger kristiinavaik jordanambra serenitysoftware thisisme-dev viktor-zireael jeffkit delhivery pmarcely yuanzhecsyz euphorbium retrography stan2133 greatspider kazitanvirahsan f0ursqu4r3 yegorich truckydev shivakumarjakkani cutso saadsahibjan meersau jordanbuchman ephemerr sp1tf1r3 ezc 9590 politeauthority donedeal-johnclaro pezo19 dt3310321 lattis mfe5003 quorumus rikiw tolorun mango233 shoaibjdev tmongero pporadzisz dealsumm eric-with-a-c wisotzky b2220333 suramyag1 steveharrison82 simon-campbell sonaysevik ehbraheem elebumm zhen-ao yotinny wahello eisroot craig-garzella maykinmedia konstantin-alphapet kennycdw rsest jonasfsilva terryhwt kobbled kvonkoni xtresoft oriolpiera caripay kevin-catlikepuma activedecay hdra girishf15 collaterate lucifer-x serafin-tech o2eo influx6-prive ehn-pkalinowski mariotutal

dicttoxml's Issues

Support Decimal

>>> import dicttoxml
>>> from decimal import Decimal
>>> mydict = dict(mydecimal=Decimal('1.2345'))
>>> dicttoxml.convert_dict(mydict, False, 'objects', True)
TypeError: Unsupported data type: 1.2345 (Decimal)

name 'some_dict' is not defined

i don't know anything but i can't convert files with this when i put path it tells me invalid syntax how do you put the path of the file to convert

Option to omit the xml declaration but keep the root or custom_root

The generated XML is hard-coded to include a declaration <?xml version="1.0" encoding="UTF-8" ?> (which is well-formed and totally legit). Nonetheless, some cases, for instance, I would need to embed the generated XML inside another XML, I would prefer not having that line of declaration.

dicttoxml/dicttoxml.py

Line 390 in 2016fe9

addline('<?xml version="1.0" encoding="UTF-8" ?>')

If I use root=False, then the whole root element and the declaration will be omit. But I would love to keep the root or custom root to make the generated code well-formed.

Thanks.

TypeError line 333

Thank for your job :)

Work perfectly with a dictionnary but I have an error with json.
here my sample code :

import json
import dicttoxml

myTestDict = {}
myTestDict["top"] = "top"
myTestDict["mid"] = []
myMidTest1 = {}
myMidTest1["mid1"] = {}
myMidTest1["mid1"]["temptext1"] = "lkjhlsdhflenlztnlkg"
myMidTest1["mid1"]["tempint1"]  = '405659'
myMidTest2 = {}
myMidTest2["mid2"] = {}
myMidTest2["mid2"]["temptext2"] = "lkjhlzkelklzeikuvlivlih"
myMidTest2["mid2"]["tempint2"]  = '98765321354685'
myTestDict["mid"].append(myMidTest1)
myTestDict["mid"].append(myMidTest2)
		
xml = dicttoxml.dicttoxml(myTestDict)
print(xml)
		
jsonTest = json.dumps(myglobaldict )
xml = xmlutility.dicttoxml(jsonTest)
print(xml)

Traceback :

line 177, in convert : "return convert_kv(item_name, obj, attr_type, cdata)"
line 333, in convert_kv : "attr['type'] = get_xml_type(val)"
TypeError: 'bool' object does not support item assignment


Next, 
Have you push the code to revert ?
from xml create a json ar dict ?

Thanks again for you job :)

Include README.markdown and LICENCE.txt in tarballs

I've posed dicttoxml for inclusion in Fedora and EPEL (https://bugzilla.redhat.com/show_bug.cgi?id=1084199), and they'd like the licence included in the tarball so that there's more of a guarantee that the LICENSE file for a given release agrees with the actual license for that release. Would it be possible to include LICENSE.txt (and, ideally, README.markdown) in the release tarballs in the future?

Thanks!

repeating tags

I would like to create repeating tags xml

<root>
   <head></head>
   <data></data>
  <data></data>
</root>

is this possible?

Fails on integer key names.

When using an integer as a key name, fails to convert to XML.

Multiple elements of the same name?

I need to generate some XML from a Python dictionary that has multiple elements of the same name.

<country>
    <city>Chicago</city>
    <city>Boston</city>
</country>

Obviously the following won't work. I assumed it would produce a syntax error, but it simply hides one of the keys:

>>> from dicttoxml import dicttoxml
>>> dicttoxml({'country':{'city':'Chicago', 'city': 'Boston'}}, attr_type=False, root=False)
'<country><city>Boston</city></country>'

I tried a list, but that wraps each list element in an item element:

>>> dicttoxml({ 'country': [ {'city': 'Chicago'}, {'city': 'Boston'} ] }, attr_type=False, root=False)
'<country><item><city>Chicago</city></item><item><city>Boston</city></item></country>'

Here's what I want the output to be

'<country><city>Chicago</city><city>Boston</city></country>'

What dictionary and/or function call settings (e.g. item_func=None) would produce that?

Feature Requests

Feature requests:

Add a root argument to dict2xml() with a default value of True. If it is set to false, the function does not wrap the generated XML in an XML declaration and a <root> ... </root> element.
Allow dict2xml() to consume any supported data type, not just a dict, so you could, e.g. feed a list into the function and it would return XML.

Is there a way to not include the item tag in a list?

Issue raised on StackOverflow

http://stackoverflow.com/questions/25104385/error-in-python-generated-xml/25105745

codec error

a

{'body': '\xe6\xb4\xbb\xe5\x8a\xa8\xe8\xae\xa2\xe5\x8d\x95', 'mch_id': '10075693', 'out_trade_no': 'iT201ac2FuqnlLpgq0rtM74mh8MCL81b', 'total_fee': 888, 'appid': 'wxb16fc7aa939867f6', 'notify_url': 'http://wagon.cn/place_order_success', 'spbill_create_ip': '0.0.0.0', 'nonce_str': 'gxA4XI1gMAALItAHQlzaUoNIE4xCYA8N', 'trade_type': 'JSAPI', 'sign': '2D2BF7CD5263D1113E1AB5601059AF82'}

dicttoxml.dicttoxml(a)

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/dicttoxml.py", line 257, in dicttoxml
addline('<%s>%s</%s>' % (custom_root, convert(obj, ids, attr_type, parent=custom_root), custom_root))
File "/usr/local/lib/python2.7/dist-packages/dicttoxml.py", line 134, in convert
return convert_dict(obj, ids, parent, attr_type)
File "/usr/local/lib/python2.7/dist-packages/dicttoxml.py", line 145, in convert_dict
LOG.info('Looping inside convert_dict(): key="%s", val="%s", type(val)="%s"' % (key, val, type(val).name))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)

Checking int the "convert_list" function.

elif type(item) in (list, set, tuple):
May be this checking is too narrow? It will be cool to make it more wide.
For example, let it to use any iterable Python-object.

Boolean turns into number

Hi,

Ref this:

Python -> XML
integer int
long long
float float
Decimal number
string str
unicode str
datetime str
None null
boolean bool
list list
set list
tuple list
dict dict

This code produces the below xml. the element has a type attribute set to "number". Should be bool, if I am reading the above correctly and haven't missed something obvious.

from dicttoxml import dicttoxml

aBool = bool(1) 
aString = str("Bob")
anInt = int(10)

data = [
   { "person": {
   "name" : aString,"age":anInt, "kid": aBool}
   }
   ]

xml = dicttoxml(data,attr_type = True, root=False) 
xml_string = xml.decode('utf-8')

print(xml_string)

<item type="dict">
    <person type="dict">
        <kid type="number">True</kid>
        <name type="str">Bob</name>
        <age type="int">10</age>
    </person>
</item>

Empty tag is not returned with both sides

Consider and empty key like {''experience": ''}
program returns <experience/>
Instead of <experience></experience>

How can i return double tags when value is null?

License question

Hi,

Quick question - does the GNU GPL2 mean any scripts on github for example that utilize this library also have to be published under the GPL2?

I tend to release my stuff under a custom BSD derived license which makes them free to use, copy and modify and don't want to go back to GPL licensing my works so I'm inclined to not use a GPL library if it takes away my own choice of license on my own works.

Is it possible to license this under the LGPL so that it only applies to this library itself?

Thanks

Hari

dicttoxml(None) and dicttoxml("name") break in 1.7

In 1.6.6:

In [1]: from dicttoxml import dicttoxml

In [2]: dicttoxml(None)           
Out[2]: b'<?xml version="1.0" encoding="UTF-8" ?><root><item type="str"></item></root>'

In [3]: dicttoxml("name")
Out[3]: b'<?xml version="1.0" encoding="UTF-8" ?><root><item type="str">name</item></root>'

In 1.7.4:

In [1]: from dicttoxml import dicttoxml

In [2]: dicttoxml(None)                
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-d2c15ddde005> in <module>()
----> 1 dicttoxml(None)

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in dicttoxml(obj, root, custom_root, ids, attr_type, item_func, cdata)
    391         addline('<%s>%s</%s>' % (
    392         custom_root,
--> 393         convert(obj, ids, attr_type, item_func, cdata, parent=custom_root),
    394         custom_root,
    395         )

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in convert(obj, ids, attr_type, item_func, cdata, parent)
    184 
    185     if obj is None:
--> 186         return convert_none(item_name, '', attr_type, cdata)
    187 
    188     if isinstance(obj, dict):

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in convert_none(key, val, attr_type, attr, cdata)
    361 
    362     if attr_type:
--> 363         attr['type'] = get_xml_type(val)
    364     attrstring = make_attrstring(attr)
    365     return '<%s%s></%s>' % (key, attrstring, key)

TypeError: 'bool' object does not support item assignment

In [3]: dicttoxml("name")              
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-0cf4e76abe5a> in <module>()
----> 1 dicttoxml("name")

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in dicttoxml(obj, root, custom_root, ids, attr_type, item_func, cdata)
    391         addline('<%s>%s</%s>' % (
    392         custom_root,
--> 393         convert(obj, ids, attr_type, item_func, cdata, parent=custom_root),
    394         custom_root,
    395         )

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in convert(obj, ids, attr_type, item_func, cdata, parent)
    175 
    176     if isinstance(obj, numbers.Number) or type(obj) in (str, unicode):
--> 177         return convert_kv(item_name, obj, attr_type, cdata)
    178 
    179     if hasattr(obj, 'isoformat'):

/usr/local/lib/python3.4/dist-packages/dicttoxml.py in convert_kv(key, val, attr_type, attr, cdata)
    331 
    332     if attr_type:
--> 333         attr['type'] = get_xml_type(val)
    334     attrstring = make_attrstring(attr)
    335     return '<%s%s>%s</%s>' % (

TypeError: 'bool' object does not support item assignment

item_func usage - end tag mismatch

I'm trying to change a list of dicts to xml. In the xml I need the item name to be 'module' instead of the default 'item' so have been trying to use the new item_func argument to dicttoxml to achieve this. Example of my list:
[{"key1": "1", "key2": "2", "key3": "3"}, {"key1": "1", "key2": "2", "key3": "3"}]

The following code nearly works but the module/item tags are incorrect i.e. the resulting xml has opening 'module' tags but closing 'item' tags:

import dicttoxml
a = [{"key1": "1", "key2": "2", "key3": "3"}, {"key1": "1", "key2": "2", "key3": "3"}]
f = lambda x: 'module'
output = dicttoxml.dicttoxml(a, attr_type=False, custom_root='xml', item_func=f)
print(output)
<?xml version="1.0" encoding="UTF-8" ?><xml><module><key3>3</key3><key2>2</key2><key1>1</key1></item><module><key3>3</key3><key2>2</key2><key1>1</key1></item></xml>

(May be easier to spot issue from xml output substring: <module><key3>3</key3><key2>2</key2><key1>1</key1></item> - see start 'module' tag & end 'item' tag)

Is this a bug in item_func? Or is my use case just not supported?

Thanks!

Invalid links in pypi distribution

The links page https://pypi.python.org/simple/dicttoxml/ contains broken build_url links, e.g.:

https://github.com/quandyfactory/dicttoxml/blob/master/dist/dicttoxml-1.5.7.tar.gz?raw=true

This confuses some build tools such as pants (http://pantsbuild.github.io/), which parse the link content.

Support for custom attributes

Maybe this can already be done, but I haven't seen it. Can insert custom attributes other than "type" and "id"?

The reason I bring this up is that this could very easily support converting a json file into a kml file save for that limitation.

Duplication in detecting a list, tuple, set or iterable data type

Line 69 in get_xml_type() reads:

if type(val).__name__ in ('list', 'set', 'tuple') or isinstance(val, collections.Iterable):

This is redundant. List, set and tuple types are also instances of collections.Iterable. So the line can be changed to:

if isinstance(val, collections.Iterable):

Same with line 127 in convert():

if type(obj) in (list, set, tuple) or isinstance(obj, collections.Iterable):

It can be changed to:

if isinstance(obj, collections.Iterable):

Same with line 158 in convert_dict():

elif type(val) in (list, set, tuple) or isinstance(val, collections.Iterable):

It can be changed to:

elif isinstance(val, collections.Iterable):

Same with line 193 in convert_list():

elif type(item) in (list, set, tuple) or isinstance(item, collections.Iterable):

It can be changed to:

elif isinstance(item, collections.Iterable):

Add support to specify a custom `item` name

Add support to specify a custom item name.
e.g.:

<?xml version="1.0" encoding="UTF-8" ?>
<root>
    <my_list>
        <my_custom_item_name>
            <foo>Abc</foo>
            <bar>123</bar>
        </my_custom_item_name>
        <my_custom_item_name>
            <foo>Def</foo>
            <bar>456</bar>
        </my_custom_item_name>
    </my_list>
</root>

instead of:

<?xml version="1.0" encoding="UTF-8" ?>
<root>
    <my_list>
        <item>
            <foo>Abc</foo>
            <bar>123</bar>
        </item>
        <item>
            <foo>Def</foo>
            <bar>456</bar>
        </item>
    </my_list>
</root>

An error while converting lists containing 'None' elements

The code:

dicttoxml([None])

produces:

TypeError: Unsupported data type: None (NoneType)

I belive it should return:

u'<?xml version="1.0" encoding="UTF-8" ?><root><item type="null"></item></root>'

Parse XMLs created with dicttoxml

XMLs created with your package solve a lot of the issues I encountered parsing/unparsing XMLs created using the xmltodict package (e.g. inferring data type, parsing empty/single element lists, etc.)

I need the ability to run this in reverse; to parse the XMLs I create and turn them back into dicts.

Would you accept a PR for this?

Proposal: Add namespace and attribute support

Namespaces are definitely a needed feature, as is an easier way to add attributes to tags. I propose this be done in the following way:

from dicttoxml import dicttoxml

dict = {
    "Envelope": {
        "_ns":    "soapenv",
        "_attrs": {
            "soapenv": { "_ns": "xmlns", "value": "http://schemas.xmlsoap.org/soap/envelope/" },
            "urn": { "_ns": "xmlns", "value": "urn:partner.soap.sforce.com" },
            "otherExample": "test value"
        },
        "Header": {
            "_ns":    "soapenv",
            "SessionHeader": {
                "_ns": "urn",
                "sessionId": "3MVG9uudbyLbNPZMtuxO54qpe.DK5jaS6q_pb7mBcusBAzQcItE_a2ZZoOSucuDuYqLgWgFd6iHKr6GUAWjWx"
            }
        },
        "Body": {
            "_ns": "soapenv",
            "query": {
                "_ns": "urn",
                "queryString": {
                    "_ns": "urn",
                    "text": "SELECT Id, Name FROM Account LIMIT 2"
                } 
            }
        }
    }
}

myXml = dicttoxml(dict, root=False, attrType=False)
print(myXml)

Which would return the following XML structure:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
   xmlns:urn="partner.soap.sforce.com" otherExample="test value">
  <soapenv:Header>
     <urn:SessionHeader
    <urn:sessionId>3MVG9uudbyLbNPZMtuxO54qpe.DK5jaS6q_pb7mBcusBAzQcItE_a2ZZoOSucuDuYqLgWgFd6iHKr6GUAWjWx</urn:sessionId>
     </urn:SessionHeader>
  </soapenv:Header>
  <soapenv:Body>
     <urn:query>
        <urn:queryString>SELECT Id, Name FROM Account LIMIT 2</urn:queryString>
     </urn:query>
  </soapenv:Body>
</soapenv:Envelope>

I could take a crack at implementing this myself, but I am by no means a great python developer yet. This is just a feature I feel is sorely needed and would put this library ahead of all the rest. Thanks!

converting python dict to xml taking longer time and sometime hanged the system

while trying to write python object to XML it's taking a long time or some time even after 4-5 minutes, there is no response from the library.

Add support to a custom root element.

Add support to a custom root element.
e.g.:

< customroot > < foo >Abc< /foo > < bar >123< /bar > < /customroot >

How to form a nested xml if the dict contains several list

I have this problem, I need to create someting like this:

<root>
    <item1>value</item1>
    <item2>value</item2>
    <item3>value</item3>
    <item4>
         <i4>value</i4>
         <i4>value1</i4>
         <i4>value2</i4>
    </item4>
    <item5>
         <i5>value</i5>
         <i5>value1</i5>
         <i5>value2</i5>
    </item5>
</root>

item4 is a list and i4 is a dict.

Mostly it works fine, if I set:

my_item_func = lambda x: 'i4'
xml = dicttoxml.dicttoxml(form, item_func=my_item_func)

And the key is named 'item4' I have the desired results.

The problem is that the items in the second list i5 are named also i4, as I can set only one function to name the items.

I tried to set the item func as:

my_item_func = lambda x: x
But I get item4 and item5 instead of i4 and i5.

There is any way to get the desired results.

Thanks a lot.

AttributeError when using attr_type=False in version 1.7.4

attempting to convert json to XML with custom root and attr_type set leads to the the below:

File "/opt/code/bsc/monitor/listener.py", line 140, in getHealth
xml_resp = dicttoxml(json_resp, custom_root='esc_health_report', attr_type=False)
File "/usr/lib/python2.6/site-packages/dicttoxml.py", line 393, in dicttoxml
convert(obj, ids, attr_type, item_func, cdata, parent=custom_root),
File "/usr/lib/python2.6/site-packages/dicttoxml.py", line 177, in convert
return convert_kv(item_name, obj, attr_type, cdata)
File "/usr/lib/python2.6/site-packages/dicttoxml.py", line 334, in convert_kv
attrstring = make_attrstring(attr)
File "/usr/lib/python2.6/site-packages/dicttoxml.py", line 117, in make_attrstring
attrstring = ' '.join(['%s="%s"' % (k, v) for k, v in attr.items()])
AttributeError: 'bool' object has no attribute 'items'

dicttoxml generates INFO logs

I am using dicttoxml to convert some dictionary data into xml & It works fine. However, it prints every action in the logs. We always have to see the logs file to debug any issue we encounter during our testing. These logs are not at all useful for any other developer if the dictionary gets successfully converted into XML.

I would be grateful if you could disable any type of logging as a default action. However, developer should be able to enable/disable logging using set_debug method.

Thanks a lot for amazing python module :)

dicttoxml_logs.log

Deprecation warning for collections abstract base classes

I'm getting a deprecation warning that says the package will not work after Python 3.0 since collections abstract base classes are moving from collections to collections.abc

Bug: dict value with unsupported type fails silently

If you pass in a dict and one of the values is an unsupported type (e.g. a datetime object), the code doesn't raise an exception - it just passes silently over the value without adding it to the XML output. Here's an example that produces the issue:

>>> import dict2xml
>>> import datetime
>>> now = datetime.datetime.today()
>>> mydict = { 'name': 'Joe Blow', 'time': now }
>>> xml = dict2xml.dict2xml(mydict)
>>> print xml
<?xml version="1.0" encoding="UTF-8" ?><root><name type="str">Joe Blow</name></root>
>>> # the 'time' key and its datetime value just silently disappeared

I propose the following fix:

Raise an exception in convert_dict() and convert_list() if one of the elements is an unsupported type (currently, only convert() raises an exception).
Include an optional parameter that will automatically convert datetimes into UTC strings in the XML output.
Include an optional parameter that will automatically convert other unsupported types into their string representations.

Support for CustomCode-Types?

Hi,

I'm using the library for quite a while and it is really cool. Unfortunately, I do have a project now which uses Enums and dataclasses.

The following code describes briefly what I'm doing:

from enum import Enum, auto
from dataclasses import dataclass, field, asdict
from dicttoxml import dicttoxml

class MyEnum:
    A = auto()
    B = auto()
    C = auto()

@dataclass
class MyDc
    x: str = field()
    y: int = field()
    z: MyEnum = field()

dc = MyDc()
dt = asdict(dc)
xml = dicttoxml(dt)

Creating a json file from the dictionary is possible, when using a customized JSONEncoder instance, which is handed over to the encode function.

With this issue I suggest the extension of the dicttoxml method:

def dicttoxml(obj, root=True, custom_root='root', ids=False, attr_type=True,
    item_func=default_item_func, cdata=False, *, converter_func=convert)

This way, a custom converter function can be used to serialize custom data types like Enums or special objects:

def convert_with_enum(obj, ids, attr_type, item_func, cdata, parent='root'):
    if isinstance(obj, Enum):
          item_name = item_func(parent)
          return convert_kv(item_name, obj.name, attr_type, cdata)
    return convert(obj, ids, attr_type, item_func, cdata, parent)

I can submit a pull request, if you consider this issue as valueable.

Regards Carsten

Possible corrupt 1.3.1 release?

I have successfully been installing v1.3.1 of dicttoxml up until today via easy_install as part of a build job.

However, it seems that the release might have gotten corrupted:

Searching for dicttoxml
Reading http://pypi.python.org/simple/dicttoxml/
Best match: dicttoxml 1.3.1
Downloading https://github.com/quandyfactory/dicttoxml/blob/master/dist/dicttoxml-1.3.1.tar.gz?raw=true
Processing dicttoxml-1.3.1.tar.gz
error: Not a recognized archive type: /tmp/easy_install-MtVemN/dicttoxml-1.3.1.tar.gz

It's very likely that this was caused by GitHub itself... Maybe you could delete the release and make it available again?

Importing ABC directly from collections was deprecated and will be removed in Python 3.10. Use collections.abc

dicttoxml.py
99:    if isinstance(val, collections.Iterable):
191:    if isinstance(obj, collections.Iterable):
235:        elif isinstance(val, collections.Iterable):
298:        elif isinstance(item, collections.Iterable):

Setting debug level prints to console

Is it possible to remove the "Debug mode is off." print statement in the console on line 42/ 49 or dicttoxml.py. I really like this module, but pride myself on not having random print statements to the console. I'm using a ton of other modules in conjunction with dicttoxml and this print statement just ends up being confusing and unnecessary. I can make a PR for this if it helps at all. Thanks!

no root and pretty print

apparently the pretty print raise an exception when in case of absence of root element

My XML tags are scrambled

I am trying to convert a json to XML. It converts successfully but my tags are scrambled. The sequence is changed during conversion.
JSON=http://paste.ubuntu.com/24923493/
Converted XML= https://pastebin.mozilla.org/9025527

When we compare with JSON it is clear that there are several elements which are changed in sequence e.g

1

1.1.1.1

0.0.0.0/0

According to JSON prefix should occur before next-hop and inside next-hop address should occur before distance...similarly a lot of other elements are also changed which causes trouble.
Could you please guide me how to cater this? I'll be obliged.

One thing more i don't want that "item" tag to be occurred. I can only change its name but cannot remove it. How to disable this?

Conversion fails when JSON key name is not a legal XML element name

Given the input JSON:

{
   "^.{0,256}$": "foo"
}

I receive the following error:

dcramer@anatine ~/Desktop/json2plainparams/json_schema (master)
$ ./json2xml.py test.json 
Debug mode is on. Events are logged at: dicttoxml.log
test.json
Traceback (most recent call last):
  File "./json2xml.py", line 18, in <module>
    dom = parseString(xml)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 1931, in parseString
    return expatbuilder.parseString(string)
  File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 940, in parseString
    return builder.parseString(string)
  File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 46

Due to the fact that <^.{0,256}$>foo</^.{0,256}$> would not be well-formed xml.

I would suggest an xml format like foo to avoid this problem.

to much logging info

there is too much logging info in the log file who use the source file.
please remove the LOG object.

2020-01-17 11:23:04,669 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="appid", val="wxfe6689008f52e1d4", type(val)="str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "appid" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "appid"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="appid", val="wxfe6689008f52e1d4", type(val) is: "str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "appid" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "appid"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="mch_id", val="1570462241", type(val)="str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "mch_id" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "mch_id"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="mch_id", val="1570462241", type(val) is: "str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "mch_id" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "mch_id"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="nonce_str", val="689c2e4e08764999940e649d2663a446", type(val)="str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "nonce_str" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "nonce_str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="nonce_str", val="689c2e4e08764999940e649d2663a446", type(val) is: "str"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "nonce_str" with attr "{}"
2020-01-17 11:23:04,670 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "nonce_str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="sign_type", val="MD5", type(val)="str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "sign_type" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "sign_type"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="sign_type", val="MD5", type(val) is: "str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "sign_type" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "sign_type"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="out_trade_no", val="2020011711224600003020110095", type(val)="str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "out_trade_no" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "out_trade_no"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="out_trade_no", val="2020011711224600003020110095", type(val) is: "str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "out_trade_no" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "out_trade_no"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-209: Looping inside convert_dict(): key="sign", val="5EB02FB256A94478A7E64AD85C9047D6", type(val)="str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "sign" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "sign"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-327: Inside convert_kv(): key="sign", val="5EB02FB256A94478A7E64AD85C9047D6", type(val) is: "str"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-135: Inside make_valid_xml_name(). Testing key "sign" with attr "{}"
2020-01-17 11:23:04,671 [17608] INFO dicttoxml.py-123: Inside key_is_valid_xml(). Testing "sign"

Typo in "convert_list" function

raise TypeError('Unsupported data type: %s (%s)' % (obj, type(obj).name))
There is no "obj" definition in this function.

Unique identifier for <item> containers

Add some way to uniquely identify consecutive tag pairs in the same nested level. As per your suggestion, an incremented tag attribute should do the trick.Should probably be optional with a flag passed to dicttoxml().

I assume the unique id will be akin to the item/array index number that the tag represents, and will reset for each nested level. (globally unique seems silly, but on that note..)

A longer term goal might be to accept an iterator from the user at the beginning of each sequence; leaving flexibility open without excessive complication. The trick might be embedding the iterator. Maybe perform a typecheck on the 0th index of any sequence, and if a custom class instance is found, pop it from the sequence and process it as an attibute value iterator.
We can let this last one ferment for a while ;) Thanks for your hard work.

Memory leak??

I executed dicttoxml.dicttoxml( ) in rapid succession on JSON data, and each execution would be slower than the previous. I am applying this to conversion of large amounts of data. So in a very short time, the conversion became too slow to finish in a reasonable time. The conversion times for 1000 records per iteration were approximately: 5s, 15s, 22s, 45s...
I think there may be a memory leak.
So, to hack around this issue, I used reload(dicttoxml) after each conversion, and the conversion times remained constant each time.
Is this issue known? Is there a better workaround available? Thanks

return str in py3

The .encode('utf-8') that is done at the very end causes the returt type to be bytes in py3.
is there a reason the dicttoxml function does't return str in py3?

'True' is not a valid value for 'boolean'

When converting boolean type from dict to xml, it is converted to 'True' instead of 'true'
According to https://www.w3.org/TR/xmlschema-2/#boolean, 'True' is not a valid value for 'boolean'

Allow : in key

According to Microsoft this is valid XML:

<mx:CurrencyCode>EUR</mx:CurrencyCode>

But dicttoxml turns

obj = { 'mx:CurrencyCode': 'EUR' }
xml = dicttoxml(obj, attr_type=False, root=False)
print(xml)

into

<key name="mx:CurrencyCode">EUR</key>

instead of

<mx:CurrencyCode>EUR</mx:CurrencyCode>

It would be great to have an option to switch that behavior off.

use of mutable object in python signatures

Examining v1.6.6 - I see 3 methods with mutable object in signature.
This is never a good idea.

def convert_kv(key, val, attr_type, attr={}):
def convert_bool(key, val, attr_type, attr={}):
def convert_none(key, val, attr_type, attr={}):

These should be changed to:
def convert_kv(key, val, attr_type, attr=None):
def convert_bool(key, val, attr_type, attr=None):
def convert_none(key, val, attr_type, attr=None):

And the following - or something similar, inserted after each LOG statement found in each method.

    attr = dict() if attr is None else attr
    if not isinstance(attr, dict):
        raise TypeError("Unexpected type: attr={a}, type={t} vs dict".format(a=attr, t=type(attr).__name__))

need process bool in right way

def test_xml_result():
        dict_data = {
            'token': None,
            'version': None,
            'create_time': 12345,
            'succeed': True,
            'messages': [],
            'data': {'A': 1}
        }
        print(dicttoxml(dict_data))

the result is

<?xml version="1.0" encoding="UTF-8" ?><root><token type="null"></token><version type="null"></version><create_time type="int">12345</create_time><succeed type="bool">True</succeed><messages type="list"></messages><data type="dict"><A type="int">1</A></data></root>

In fact, True is not the xml bool value type, it should be true.
We fixed this issue, please check the attachment file, it's the code.

dicttoxml_fix_bool_issue.txt

Library is susceptible to XML Injection

Hi,

the library fails to XML-encode the dict input and is therefore susceptible to XML injection. See this example:

>>> from dicttoxml import dicttoxml
>>> dict = {'foo': 'bar', '" injected-attribute="value" <!--': 'bar'}
>>> print(dicttoxml(dict, attr_type=False))
b'<?xml version="1.0" encoding="UTF-8" ?><root><key name="" injected-attribute="value" <!--">bar</key><foo>bar</foo></root>'

So one can change the XML structure. Suppose the application using this library and the input comes (at least partially) from the user, then this may lead to severe security vulnerabilities. See also here.

Proposed solution: XML-encode all data that is placed into the XML document.

Cheers
Thomas

Creates invalid XML when a dict has keys containing "/"

Let's say I have this JSON..

{
    "routes": {
        "/test/[test variable]": {
            "GET": "A test route"
        },
        "/routes": {
            "GET": "The list of routes in a computer-readable format"
        }
    }
}

This gives us..

<?xml version="1.0" encoding="UTF-8" ?>
<root>
    <routes type="dict">
        </test/[test variable] type="dict">
            <GET type="str">A test route</GET>
        <//test/[test variable]>
        </routes type="dict">
            <GET type="str">The list of routes in a computer-readable format</GET>
        <//routes>
    </routes>
</root>

This is invalid XML. Any ideas?

quandyfactory / dicttoxml Goto Github PK

dicttoxml's People

Contributors

Stargazers

Watchers

Forkers

dicttoxml's Issues

Recommend Projects

Recommend Topics

Recommend Org