3breadt / dd-plist Goto Github PK

By any chance is this an MIT license?

Hello,

Our application has to deal with a lot of plist files that usually come from unknown sources. Sometimes it's corrupted files but with a correct header.

When I run BinaryPropertyListParser on the attached sample file it will produce an OutOfMemoryError which sometimes can crash the entire application. It would be great to improve the validation part and throw an exception if incorrect header or trailer is detected:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at com.dd.plist.BinaryPropertyListParser.doParse(BinaryPropertyListParser.java:135)
	at com.dd.plist.BinaryPropertyListParser.parse(BinaryPropertyListParser.java:87)
	at com.dd.plist.BinaryPropertyListParser.parse(BinaryPropertyListParser.java:154)
	at com.dd.plist.BinaryPropertyListParser.parse(BinaryPropertyListParser.java:166)
	at PlistIssue.main(PlistIssue.java:10)

Code to produce:
BinaryPropertyListParser.parse(new File("sample.plist"));

Version used: 1.20

Sample file: sample.plist.zip

Thanks.

The parser / writer methods receiving a File object, creates streams and never / improperly close them, causing resource leaks.

For instance:
BinaryPropertyListParser - The FileInputStream is never closed.
BinaryPropertyListWriter - The FileOutputStream is closed, but with no try\finally block.

These open files can cause all sort of weird behaviors, such as running out of file descriptors, or inability to move/delete the file after reading/writing.

It is required (since it is Java 1.5) to use the try-finally pattern to close the streams:

    InputStream is = null;
    try {
         is = new FileInputStream(file);
         // read the file
    } finally {
         if (is != null) is.close();
    }

The Apple mdm documentation shows that profile keys need to start with uppercase, such as RemovalDate (in "date" format).
However, plist does not support specifying the key to be converted to string.
So I have to use Jackson to convert it as follows：
POJO -> Map -> NSObject -> String

public class Profile {
    @JsonProperty("RemovalDate")
    private Date removalDate;
}

Not:
POJO -> NSObject -> String

However, the 'RemovalDate' in the Map is in String format, not in Date format. It caueses a conversion error.
I got:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>RemovalDate</key>
	<string>2023-08-07T07:53:18Z</string>
</dict>
</plist>

It shuold be <date>...</date>:

	<key>RemovalDate</key>
	<date>2023-08-07T07:53:18Z</date>

Is there any solution?
Thanks.

Hi,

I got a 404 for the online Javadoc site

Thank you.
Guadaran.

https://github.com/3breadt/dd-plist/blob/master/src/main/java/com/dd/plist/XMLPropertyListParser.java#L91 is a synchronized method and is causing performance problems in our peak load testing.

This method was synchronized from the first commit. (ad6ecc4#diff-2c27b48e0134dc885800b0487859238aR71). I can understand that there are initializing the docBuilderFactory variable lazily so that was the reason that method was synchronized.

With the current code, (https://github.com/3breadt/dd-plist/blob/master/src/main/java/com/dd/plist/XMLPropertyListParser.java#L49), the factory is initialized eagerly. So, the method need not be synchronized now.

My problem is the following:

public class PayloadDto {
    private List<PayloadContentDto> payloadContent = new ArrayList<>();
}

public class PayloadContentDto {
    protected String payloadType;
}

public class A extends PayloadContentDto {
   private String aProperty;
}

public class B extends PayloadContentDto {
   private String bProperty;
}

public PayloadDto getDtoFromPListByteArray(byte[] content) {
...
    NSObject parse = PropertyListParser.parse(content);
    PayloadDto payloadDto = parse.toJavaObject(PayloadDto.class);
...
}

When calling getDtoFromPListByteArray the payloadContent list contains these types:

[
class PayloadContentDto(payloadType="aType"),
class PayloadContentDto(payloadType="bType"),
]

instead of these expected ones:

[
   class A(payloadType="aType", aProperty="a's property"),
   class B(payloadType="bType", bProperty="b's property"),
]

Is there any way to achive the proper class convertion?

Something like Jackson annotations

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "payloadType", visible = true)
@JsonSubTypes({
        @JsonSubTypes.Type(value = A.class, name = "aType"),
        @JsonSubTypes.Type(value = B.class, name = "bType"),
})

Please check if the name can be deleted?

NSKeyedArchiver

When parsing errors occur there is just a message that something did not work out. For users it is good to know where (filename, line, column). I know it is not always possible, e.g. when some binary plist is loaded via the network as an InputStream.
But in cases where a simple text file is parsed the likelyhood of this being inspected manually is a lot higher.

Would it be possible to add a 'location' field to NSObject that contains as much data as was available when creating it?

Part of Issue #74

An invalid binary property list may contain a NULL object as a key of an NSDictionary. This should cause a PropertyListFormatException instead of a NullReferenceException.

Follow up run with the changes for #73. I split up the files into folders by exception type since there are more files this time due to a longer run.

The slow folder contains files which can take up to several minutes before crashing. One file will not crash. Presumably those files trigger a long running loop somewhere.

plist-crash2.zip

I want to parse a valut to float type.
And the value is between 0~1，so, I use a float value to store this value.
But when the value is 1, it will throw an exception Cannot map NSNumber to float.
In fact, I can not make a Variable that both in type int and float.
So, I think maybe we should change the if condition in (NSObject: deserializeNumber) function

I'm parsing some InfoPlist.strings files that are in "simple string file format". For example:

/* A human readable InfoPlist.strings in simple strings format */

/* The bundle name */
"CFBundleName" = "(Base) Name";
/* The bundle display name */
"CFBundleDisplayName" = "(Base) DisplayName";
/* The short version */
"CFBundleShortVersionString" = "(Base) ShortVersion";

This is basically an ASCII NSDictionary, with an implied open/close curly brace. Is there any plan to support this? Alternatively, can you suggest any way I could augment the parser to support this? I'd be happy to submit a PR, if you can point me in the right direction.

The Simple Strings File Format is described here:
https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/LoadingResources/Strings/Strings.html

Hello, and thank you for maintaining this library.

I recently encountered an issue parsing a plist contained in an iOS app. iOS creates a CodeResources file in XML Plist format, which has dictionary with an entry for each file contained in the app. The keys are the names of the files, and the values are another dictionary with file hash data for the file. For example:

...
<key>files2</key>
<dict>
	...
	<key>Base.lproj/Main.storyboardc/MainController.nib</key>
	<dict>
		<key>hash</key>
		<data>
		H4su9pQU+nD/V4ppfPwJGSNcjv8=
		</data>
		<key>hash2</key>
		<data>
		8MhWcI9M79dqHp4tTbcUBwJCeu63sn/ekQKdGAVQLUM=
		</data>
	</dict> 
	...

The data element is parsed by the Base64 encoder included in the library, which attempts to detect whether the data is in gzip format, and if so, pipes the output through a GZipInputStream. It detects whether the data is in gzip format by checking to see if the first two bytes are equal to the gzip magic number. Unfortunately, given that the hash data is effectively just a random byte array, there is a 1 in 2^16 chance that the first two bytes of the hash will happen to be equal to the gzip magic number. The "H4su9pQU+nD/V4ppfPwJGSNcjv8=" hash value above exhibits that behavior. The decoded hexdump of that byte array is

1f 8b 2e f6 94 14 fa 70 ff 57 8a 69 7c fc 09 19
23 5c 8e ff

As you can see, the first two bytes happen to be the little-endian representation of the gzip magic number: 0x8b1f. However, this isn't actual zip data, and it results in the following error:

java.util.zip.ZipException: Unsupported compression method
	at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:169)
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
	at com.dd.plist.Base64.decode(Base64.java:1315)
	at com.dd.plist.Base64.decode(Base64.java:1266)
	at com.dd.plist.NSData.<init>(NSData.java:60)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:224)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:203)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:203)
	at com.dd.plist.XMLPropertyListParser.parse(XMLPropertyListParser.java:181)
	at com.dd.plist.XMLPropertyListParser.parse(XMLPropertyListParser.java:142)
	at com.dd.plist.PropertyListParser.parse(PropertyListParser.java:242)
	at com.dd.plist.PropertyListParser.parse(PropertyListParser.java:219)

I'd like to be able to disable the automatic zip decoding of data values, but it's not possible via the current APIs. I'd like to propose either disabling this behavior by default -- this is the default behavior for write, and should probably be symmetrical with read. Alternatively, we could plumb in a way to disable this behavior by adding an overloaded version of the PropertyListParser.parse(...) methods and similarly the XXXPropertyListParser implementations and the NSData(String) constructor. Yet another alternate implementation would be to do further examination of the byte array structure to determine if it contains a well formed zip header data, though this would only reduce the likelyhood of the issue. If you would be willing to entertain any of these fixes, or suggest an alternate, I would be happy to submit a pull request with the implementation.

Thank you again for maintaining this library. I'd be happy to lend a hand.

Using version 1.26, downloaded from Maven Central.

The NSNumber parser throws an exception when it encounters the values "+infinity" or "-infinity". This can be verified with the following code snippet:

public static void main(String[] args) {
    var a = new NSNumber("3");
    try {
        var b = new NSNumber("+infinity");
    } catch (Exception e) {
        System.out.println(e);
    }
    try {
        var c = new NSNumber("-infinity");
    } catch (Exception e) {
        System.out.println(e);
    }
}

Output:

java.lang.IllegalArgumentException: The given string neither represents a double, an int nor a boolean value.
java.lang.IllegalArgumentException: The given string neither represents a double, an int nor a boolean value.

This causes XMLPropertyListParser.parse(...) to fail when the input contains either <real>+infinity</real> or <real>-infinity</real> but I believe both of these are valid values.

Part of issue #74.

The offset and length of data to be read from a binary property list is not always checked before trying to parse objects. This results in various undocumented exceptions, like ArrayIndexOutOfBoundsException, StringIndexOutOfBoundsException or IllegalArgumentException.

I started fuzzing this library with https://github.com/CodeIntelligenceTesting/jazzer after originally fuzzing another library and seeing there some crashed in this projects code.

To reproduce the errors use the PropertyListParser.parse method.

The fuzzing treats all non declared exceptions as failure.
One file triggers the exception in BinaryPropertyListParser.doParse. Since normally no one would catch this exception I included it since changing this to a PropertyListFormatException could be reasonable.

plist-crash.zip

Part of Issue #74

NSSet.add(null) throws a NullReferenceException instead of an IllegalArgumentException.

Also parsing a binary property list where such a null object is encoded as part of an NSSet runs into that exception instead of throwing a PropertyListFormatException.

To support adding NULL objects to an NSSet a new type NSNull (compare https://developer.apple.com/documentation/foundation/nsnull) could be introduced.

The attached 1.68KB plist file causes a 4GB int[] array allocation when parsed using dd-plist-1.23. It is possibly corrupted, but I think the library should defend itself against this to avoid DOS attacks. This was first reported at sepinf-inc/IPED#1403

Let me know if you need some other information.

Thanks in advance.
1061134322640399597.zip

I currently use the Google Code Gradle dependency: compile 'com.googlecode.plist:dd-plist:1.16'
Any plans to add Gradle support to these GitHub releases of dd-plist?

When I try to convert a plist to json, the structure goes to weird when It parse NSArray, I think it's the reason that it doens't implement List

Add support Java Platform Module System (JPMS), introduced in Java version 9.

This would allow to use dd-list as a module, but it would require Java version 9 instead of 8.

For FileInputstream is OK
For zFile.getInputStream(entry), Sometimes normal and sometimes is not normal
input=zFile.getInputStream(entry)
PropertyListParser.parser(input)
error:

for reason:

/**
* Reads all bytes from an InputStream and stores them in an array, up to
* a maximum count.
*
* @param in The InputStream pointing to the data that should be stored in the array.
* @return An array containing all bytes that were read from the input stream.
* @throws java.io.IOException When an IO error while reading from the input stream.
*/
protected static byte[] readAll(InputStream in) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buf = new byte[512];
int read = 512;
while (read == 512) {
read = in.read(buf);
if(read != -1)
outputStream.write(buf, 0, read);
}
return outputStream.toByteArray();
}
Will the above code truncate the input stream ? then Information is not complete.
This problem corresponding to the app : http://os-ios.liqucn.com/rj/22709.shtml

Reproduction steps:
Purposefully try to parse a badly quoted string. For example:

{
    "Hello" = "Worl\d";
}

Expected results:
The ParseException should point errorOffset to the \d, implied by the expression this.index + ex.getErrorOffset() at

Actual results:
errorOffset points to an index out of bounds because it adds ex.getErrorOffset() (the error offset within the quoted string being parsed) to this.index (the end of the quoted string being parsed).

As per
claunia/plist-cil#3
I checked and that's what Apple's code does.
tl;dr: it takes a 64-bit number and stores it in 1, 2, 4 or 8 big-endian bytes depending on its value.

So creating an UID object from a number (byte/int/long) should be supported.
Here's my code:
claunia/plist-cil@1979dab

The method PropertyListParser.parse(InputStream) is documented as

Parses a property list from an InputStream. This method does not close the specified input stream.

Unfortunately the underlying Xerxes XML parser does close the InputStream.

As a workaround I wrap the InputStream to ignore close():

      PropertyListParser.parse(
          new FilterInputStream(inputStream) {
            @Override public void close() {}
          });

I see two ways to fix this bug:

1. Update the documentation to point out that the InputStream might be closed.
2. Update the implementation to wrap the InputStream to ignore close().

<dict>
	<key>DownloadPercentComplete</key>
	<real>nan</real>
	<key>IsDownloaded</key>
	<true/>
	<key>ProductKey</key>
	<string>061-21545</string>
	<key>Status</key>
	<string>Idle</string>
</dict>

It is a MDM request from Apple Macbook, when I tried to parse it, an exception throws.

java.lang.IllegalArgumentException: The given string neither represents a double, an int nor a boolean value.
	at com.dd.plist.NSNumber.<init>(NSNumber.java:138)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:232)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:215)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:222)
	at com.dd.plist.XMLPropertyListParser.parseObject(XMLPropertyListParser.java:215)
	at com.dd.plist.XMLPropertyListParser.parse(XMLPropertyListParser.java:193)
	at com.dd.plist.XMLPropertyListParser.parse(XMLPropertyListParser.java:154)
	at com.dd.plist.PropertyListParser.parse(PropertyListParser.java:240)
	at com.dd.plist.PropertyListParser.parse(PropertyListParser.java:217)

Able to launch Billion Laugh attack (https://en.wikipedia.org/wiki/Billion_laughs_attack).

However, adding below code in XMLPropertyListParser seems to restrict the vulnerability.
FACTORY.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);

Please review.

Part of Issue #74.

NSArray, NSDictionary and NSSet contain references to child objects. If a binary property list is malformed it may contain a reference to itself or a parent NSDictionary/Array/Set. This causes a StackOverflowException in the recursive BinaryPropertyListParser.parseObject method.

Such cyclic references should be detected and parsing be aborted.

Hi!

I'm parsing binary plist and get something like that:

        <dict>
            <key>$class</key>
            <string>7d0ffffffe3</string>
            <key>NS.bytes</key>
            <data>
                9jz+91kt1WmqSyfQEvv/2eDtiF0=
            </data>
        </dict>

What is a <string> format?
<data>is a base64 format.

com.dd.plist.propertyListFormatException the given data is not a peoperty list of a supported

The two methods com.dd.plist.PropertyListParser.determineType(InputStream, int) and com.dd.plist.PropertyListParser.determineType(byte[]) seem to be hard-coded to the UTF-8 BOM EF BB BF.

However the parser implementation of ASCIIPropertyListParser is for example also able to handle UTF-16 and UTF-32 files, but you will never get to that point if you try to read an UTF-16 ASCII file using one of the com.dd.plist.PropertyListParser.parse(..) method as because of the BOM the determineType(String) will not work correctly so that you end up with a PropertyListFormatException.

Apple split NSdata base 64 string into multiple lines not exceeding 76 characters (including 8 whitespace-per-tab indentation).

Also, the indentation of data should be in the same level of the <data> tag.

See the patch:

From 262dccf3c55d055f7bf333b7f8e79e64358a33ca Mon Sep 17 00:00:00 2001
Date: Sun, 11 Nov 2018 11:41:40 +0200
Subject: [PATCH] Make XML output more compliant with Apple's format: NSData -
 Lines of data must be 76 characters max (unless indentation is too deep), and
 data indentation equals <data> tag indentation. toXMLPropertyList() - append
 new line at the end of XML string.

---
 src/main/java/com/dd/plist/NSData.java   | 11 +++++++++--
 src/main/java/com/dd/plist/NSObject.java |  4 +++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/src/main/java/com/dd/plist/NSData.java b/src/main/java/com/dd/plist/NSData.java
index 6bad76a..2a12b35 100644
--- a/src/main/java/com/dd/plist/NSData.java
+++ b/src/main/java/com/dd/plist/NSData.java
@@ -145,8 +145,15 @@ public class NSData extends NSObject {
         xml.append("<data>");
         xml.append(NSObject.NEWLINE);
         String base64 = this.getBase64EncodedData();
-        for (String line : base64.split("\n")) {
-            this.indent(xml, level + 1);
+        int indent = (level > 8) ? 8 : level;
+        int maxLen = 76 - indent * 8;
+
+        // Lines of data must be 76 characters max (unless indentation is
+        // too deep), and data indentation equals <data> tag indentation,
+        // to be compliant with Apple's format:
+        for (String line :
+            base64.split("(?<=\\G.{"+maxLen+"})")) {
+            this.indent(xml, level);
             xml.append(line);
             xml.append(NSObject.NEWLINE);
         }
diff --git a/src/main/java/com/dd/plist/NSObject.java b/src/main/java/com/dd/plist/NSObject.java
index 5a450df..8eba28c 100644
--- a/src/main/java/com/dd/plist/NSObject.java
+++ b/src/main/java/com/dd/plist/NSObject.java
@@ -100,7 +100,9 @@ public abstract class NSObject implements Cloneable {
                 .append("<plist version=\"1.0\">")
                 .append(NSObject.NEWLINE);
         this.toXML(xml, 0);
-        xml.append(NSObject.NEWLINE).append("</plist>");
+        xml
+            .append(NSObject.NEWLINE).append("</plist>")
+            .append(NSObject.NEWLINE);
         return xml.toString();
     }
 
-- 
2.17.1 (Apple Git-112)

While converting a NSDictionary to a Java Object, the type NSData is not handled in deserialisation:

java.lang.IllegalArgumentException: Cannot process NSData
        at com.dd.plist.NSObject.toJavaObject(NSObject.java:319)
        at com.dd.plist.NSObject.deserializeObject(NSObject.java:356)
        at com.dd.plist.NSObject.toJavaObject(NSObject.java:316)
        at com.dd.plist.NSObject.toJavaObject(NSObject.java:184)

It seems that the NSObject.toJavaObject does not handles NSData case. Is this intended beharvior or is this case missing?

Library Version: 1.23

There doesn't appear to be an easy way to deep clone NSDictionary or NSObject

Part of issue #74.

The UID class does not implement java.lang.Comparable.

I encountered an empty plist file in old ASCII format:

/* Localized versions of Info.plist keys */

If you use dd-plist to parse such a file you end up with the following exception:

java.lang.ArrayIndexOutOfBoundsException: 45
	at com.dd.plist.ASCIIPropertyListParser.accept(ASCIIPropertyListParser.java:267)
	at com.dd.plist.ASCIIPropertyListParser.skipWhitespacesAndComments(ASCIIPropertyListParser.java:353)
	at com.dd.plist.ASCIIPropertyListParser.parse(ASCIIPropertyListParser.java:430)
	at com.dd.plist.ASCIIPropertyListParser.parse(ASCIIPropertyListParser.java:238)
	at com.dd.plist.ASCIIPropertyListParser.parse(ASCIIPropertyListParser.java:220)
	at com.dd.plist.ASCIIPropertyListParser.parse(ASCIIPropertyListParser.java:172)
	at com.dd.plist.ASCIIPropertyListParser.parse(ASCIIPropertyListParser.java:127)

If I interpret the description of such ASCII plist files correctly the file should be valid https://en.wikipedia.org/wiki/Property_list#NeXTSTEP

At least I would have expected a ParseException instead of an ArrayIndexOutOfBoundsException

BTW: I noticed that the Unit test
https://github.com/3breadt/dd-plist/blob/master/src/test/java/com/dd/plist/test/ParseTest.java#L149-L167
contains a lot of tests that are disabled as they miss the @Test annotation. Are those tests disabled intentionally or is this a bug?

Part of Issue #74

When a NSNumber in a GnuStep ASCII property list or an XML property list contains illegal characters, the parser throws an undocumented IllegalArgumentException. A PropertyListFormatException should be thrown instead.

At the moment various exceptions are documented to be thrown by the parser:

• PropertyListFormatException
• java.io.UnsupportedEncodingException
• java.text.ParseException

To improve usability of the parser, only one exception type should be thrown in case of a parsing error due to invalid property list content.

On maven central the version 1.17 has the latest code and not the code of the tag 1.17

Modern GNUstep plists have a more efficient data encoding, and it is the <[ base64data ]> format:

GNUstep source that emits the format:
https://github.com/gnustep/libs-base/blob/753c907938c2a8c4d00cf0fbe01b7e0d020f0064/Source/NSPropertyList.m#L1951-L1956 (gnustep/libs-base@9aa5d4c)

GNUstep source that parses the format:
https://github.com/gnustep/libs-base/blob/753c907938c2a8c4d00cf0fbe01b7e0d020f0064/Source/NSPropertyList.m#L1202-L1277
(gnustep/libs-base@3f2fa9d)

(I wish there is a newer documentation page for the GNUstep format, but I have been unable to locate it.)

Hi, any plans to publish version 1.7 to Maven Central?

I report an issue about parse NSNumber to float before
#49
#49

And I saw you have fixed it, but it seems you have not publish a new version to maven repo yet.
So could you help to publish a new version?

The other type is UID for binary plist, I see there is no this type in your code, I am not sure if it's compitable for it

With the version 1.19 i have this crash on Android:

Caused by: java.lang.UnsupportedOperationException: This parser does not support specification "Unknown" version "0.0"
at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:471)
at com.dd.plist.XMLPropertyListParser.(XMLPropertyListParser.java:67)