libplist's new XML parser has numerous bugs (even a segfault!) about libplist HOT 15 OPEN

Nice! Happy New Year :)
I understand that libxml2 is more robust but is has some issues. It's leaking memory and also seems to have problems in multithreaded environments for some reason. That was the reason why I chose to do the custom parsing. See the improvements in the comments of this ticket: libimobiledevice/libimobiledevice#394
I will take care of the issues. I will probably not be perfect in the end but it will be sufficient enough to parse plist data in the usual environments.

from libplist.

@nikias Maybe my point was a little confusing (due to the attempt at cute rhetoric/story), but I was actually providing a much more robust reason why you can't use libxml2: Apple also implemented a custom XML parser, and some of the things they support are simply not going to be possible to simulate using libxml2. So, game on! If I have some time I'll be contributing patches to make this parser closely match the behavior I'm observing from Apple's list files. (If the issue really were just memory leaks or something, I'd rather go fix libxml2, but it turns out that libplist really needs a custom parser ;P.)

from libplist.

(I was just talking about this with @DHowett, and he pointed out that Apple has changed the license on CFLite to MIT and open-sourced it all as part of Swift, so anyone who ends up here as part of tracking down a parser differential might consider trying to use Apple's code to try to get exactly the expected behavior.)

from libplist.

meh, always killing my 0dayz... libxml2 and apple's development teams are on par for their issue response. it couldn't possibly get any worse!!

from libplist.

Ok perhaps a more constructive comment. can we perhaps create some test units? plist really isn't nearly as complicated as xml is. this surely could be made secure (even if not fully compliant with standards)

from libplist.

I just pushed several commits that fix the first 6 issues/examples in this list.
@saurik I am not yet sure how to handle the other cases (text after the actual XML, unclosed attributes etc.) but I must really say the "libplist detection" cases are really nice haha :D
@posixninja make check does some tests. In fact I am making a test file right now that tests the issues fixed in with latest commits.

from libplist.

Added several test cases.

from libplist.

Does Python's plistlib (Python 3+) have the same issues?

from libplist.

plistlib uses expat XML parser so I doubt it has the same issues :)

from libplist.

however it will probably have the Apple-only issues listed above.

from libplist.

@nikias I use the multi-threaded environment seems to be no problem, Make some changes to the source code, I did not initialize and release multiple times

from libplist.

@nikias Can you separate the custom XML parser branch and libxml2 bugfixes branch ?

my English is not very good :(

from libplist.

@xdeng the custom xml parser works fine so I don't see any reason to switch back. There are some differences but it isn't really a problem for every day usage (like with libimobiledevice).

from libplist.

@saurik just note, apple devices can create XML that contains null character inside element value, and it's like that since iOS 3 at least.

my xmlparser is based on expat library.

also, as for issues mentioned above, have not seen any of them, except not closed attribute, which out parser can take care off (it ignores attribute in fact)

from libplist.

I ran into a big plist file
xml format
4x,xxx nodes
file size 15MB
Using old library parsing will take up very high memory 200MB
I want to try the new library now.

from libplist.