Code Monkey home page Code Monkey logo

freddy's Introduction

Why Freddy?

Parsing JSON elegantly and safely can be hard, but Freddy is here to help. Freddy is a reusable framework for parsing JSON in Swift. It has three principal benefits.

First, Freddy provides a type safe solution to parsing JSON in Swift. This means that the compiler helps you work with sending and receiving JSON in a way that helps to prevent runtime crashes.

Second, Freddy provides an idiomatic solution to JSON parsing that takes advantage of Swift's generics, enumerations, and functional features. This is all provided without the pain of having to memorize our documentation to understand our magical custom operators. Freddy does not have any of those. If you feel comfortable writing Swift (using extensions, protocols, initializers, etc.), then you will not only understand how Freddy is organized, but you will also feel comfortable using Freddy.

Third, Freddy provides great error information for mistakes that commonly occur while parsing JSON. If you subscript the JSON object with a key that is not present, you get an informative error. If your desired index is out of bounds, you get an informative error. If you try to convert a JSON value to the wrong type, you get a good error here too.

So, Freddy vs. JSON, who wins? We think it is Freddy.

Usage

This section describes Freddy's basic usage. You can find more examples on parsing data, dealing with errors, serializing JSON instances into NSData, and more in the Wiki. You can read the documentation to see the full API.

Deserialization: Parsing Raw Data

Basic Usage

Consider some example JSON data:

{
    "success": true,
    "people": [
        {
            "name": "Matt Mathias",
            "age": 32,
            "spouse": true
        },
        {
            "name": "Sergeant Pepper",
            "age": 25,
            "spouse": false
        }
    ],
    "jobs": [
        "teacher",
        "judge"
    ],
    "states": {
        "Georgia": [
            30301,
            30302,
            30303
        ],
        "Wisconsin": [
            53000,
            53001
        ]
    }
}

Here is a quick example on how to parse this data using Freddy:

let data = getSomeData()
do {
    let json = try JSON(data: data)
    let success = try json.getBool(at: "success")
    // do something with `success`
} catch {
    // do something with the error
}

After we load in the data, we create an instance of JSON, the workhorse of this framework. This allows us to access the values from the JSON data. We try because the data may be malformed and the parsing could generate an error. Next, we access the "success" key by calling the getBool(at:) method on JSON. We try here as well because accessing the json for the key "success" could fail - e.g., if we had passed an unknown key. This method takes two parameters, both of which are used to define a path into the JSON instance to find a Boolean value of interest. If a Bool is found at the path described by "success", then getBool(at:) returns a Bool. If the path does not lead to a Bool, then an appropriate error is thrown.

Use Paths to Access Nested Data with Subscripting

With Freddy, it is possible to use a path to access elements deeper in the json structure. For example:

let data = getSomeData()
do {
    let json = try JSON(data: data)
    let georgiaZipCodes = try json.getArray(at: "states","Georgia")
    let firstPersonName = try json.getString(at: "people",0,"name")
} catch {
    // do something with the error
}

In the code json.getArray(at: "states","Georgia"), the keys "states" and "Georgia" describe a path to the Georgia zip codes within json. Freddy's parlance calls this process "subscripting" the JSON. What is typed between the parentheses of, for example, getArray(at:) is a comma-separated list of keys and indices that describe the path to a value of interest.

There can be any number of subscripts, and each subscript can be either a String indicating a named element in the JSON, or an Int that represents an element in an array. If there is something invalid in the path such as an index that doesn't exist in the JSON, an error will be thrown.

More on Subscripting

JSONDecodable: Deserializing Models Directly

Now, let's look an example that parses the data into a model class:

let data = getSomeData()
do {
    let json = try JSON(data: data)
    let people = try json.getArray(at: "people").map(Person.init)
    // do something with `people`
} catch {
    // do something with the error
}

Here, we are instead loading the values from the key "people" as an array using the method getArray(at:). This method works a lot like the getBool(at:) method you saw above. It uses the path provided to the method to find an array. If the path is good, the method will return an Array of JSON. If the path is bad, then an appropriate error is thrown.

We can then call map on that JSON array. Since the Person type conforms to JSONDecodable, we can pass in the Person type's initializer. This call applies an initializer that takes an instance of JSON to each element in the array, producing an array of Person instances.

Here is what JSONDecodable looks like:

public protocol JSONDecodable {
    init(json: JSON) throws
}

It is fairly simple protocol. All it requires is that conforming types implement an initializer that takes an instance of JSON as its sole parameter.

To tie it all together, here is what the Person type looks like:

public struct Person {
    public let name: String
    public let age: Int
    public let spouse: Bool
}

extension Person: JSONDecodable {
    public init(json value: JSON) throws {
        name = try value.getString(at: "name")
        age = try value.getInt(at: "age")
        spouse = try value.getBool(at: "spouse")
    }
}

Person just has a few properties. It conforms to JSONDecodable via an extension. In the extension, we implement a throwsing initializer that takes an instance of JSON as its sole parameter. In the implementation, we try three functions: 1) getString(at:), 2) getInt(at:), and 3) getBool(at:). Each of these works as you have seen before. The methods take in a path, which is used to find a value of a specific type within the JSON instance passed to the initializer. Since these paths could be bad, or the requested type may not match what is actually inside of the JSON, these methods may potentially throw an error.

Serialization

Freddy's serialization support centers around the JSON.serialize() method.

Basic Usage

The JSON enumeration supports conversion to Data directly:

let someJSON: JSON = 
do {
    let data: Data = try someJSON.serialize()
} catch {
    // Handle error
}

JSONEncodable: Serializing Other Objects

Most of your objects aren't Freddy.JSON objects, though. You can serialize them to Data by first converting them to a Freddy.JSON via JSONEncodable.toJSON(), the sole method of the JSONEncodable protocol, and then use serialize() to convert the Freddy.JSON to Data:

let myObject: JSONEncodable = 

// Object -> JSON -> Data:
let objectAsJSON: JSON = myObject.toJSON()
let data: Data = try objectAsJSON.serialize()

// More concisely:
let dataOneLiner = try object.toJSON().serialize()

Freddy provides definitions for common Swift datatypes already. To make your own datatypes serializable, conform them to JSONEncodable and implement that protocol's toJSON() method:

extension Person: JSONEncodable {
    public func toJSON() -> JSON {
        return .dictionary([
            "name": .string(name),
            "age": .int(age),
            "spouse": .bool(spouse)])
    }
}

Getting Started

Freddy requires iOS 8.0, Mac OS X 10.9, watchOS 2.0, or tvOS 9.0. Linux is not yet supported.

You have a few different options to install Freddy.

Add us to your Cartfile:

github "bignerdranch/Freddy" ~> 3.0

After running carthage bootstrap, add Freddy.framework to the "Linked Frameworks and Libraries" panel of your application target. Read more.

Add us to your Podfile:

source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '8.0'
use_frameworks!

pod 'Freddy'

Then run pod install.

Submodules

  1. git submodule add https://github.com/bignerdranch/Freddy.git Vendor/Freddy
  2. Drag Freddy.xcodeproj into your Xcode project.
  3. Add Freddy.framework to the "Linked Frameworks and Libraries" panel of your application target.

Carthage can be used to check out dependencies and maintain Git submodule state as well.

Swift Package Manager

Add us to your Package.swift:

import PackageDescription

let package = Package(
    name: "My Nerdy App",
    dependencies: [
        .Package(url: "https://github.com/bignerdranch/Freddy.git", majorVersion: 3),
    ]
)

iOS 7

If you would like to use Freddy with iOS 7, then you will need to use a previous release of Freddy.

Setting Breakpoint Errors

It can be helpful to set breakpoints for errors when you start working with a new set of JSON. This allows you to explore the structure of the JSON when you break. In particular, you will likely want to set a breakpoint for Freddy's JSON.Error so that you can inspect what went wrong.

Here is how you can set this sort of breakpoint:

  1. Go to the Breakpoint navigator

  2. Click the "+" button in the bottom left corner

Breakpoint navigator

  1. Select "Add Swift Error Breakpoint"

Add Error Breakpoint

Now you have a breakpoint that will only trigger when a Swift error is generated. But your program will break whenever any Swift error is thrown. What if you only want to break for Freddy's JSON.Error error?

You can edit the breakpoint to add a filter:

  1. Right-click your new error breakpoint

  2. Select Edit Breakpoint...

Edit Breakpoint

  1. A window will appear with a text box for "Type"

  2. Enter JSON.Error

Error Type

And that is pretty much it! You now have an error breakpoint that will only trigger when errors of type JSON.Error are thrown. Take a look at the framework's tests for further examples of usage. The Wiki also have a lot of very useful information.

freddy's People

Contributors

a2 avatar abijlani avatar aquageek avatar cbrauchli avatar davidahouse avatar glennrfisher avatar hitsvilleusa avatar hydhknn avatar jeremy-w avatar jjmanton avatar jlyonsmith avatar justinmstuart avatar mdmathias avatar mlilback avatar mz2 avatar natechan avatar neonichu avatar pbardea avatar radex avatar rafaelnobrepd avatar randomstep avatar rcedwards avatar subdigital avatar swilliams avatar talzag avatar tomcomer avatar volodg avatar xaxis-kenan avatar zwaldowski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

freddy's Issues

Thoughts on free functions

Result.swift currently defines 3 free functions:

  • splitResults is great - I stole it for bnr/Result too (https://github.com/bignerdranch/Result/blob/master/Result/Result.swift#L78-L90), although I renamed it partitionResults. ("partition" is Haskell's name for this same operation, and I think it's a little more accurate than split.)
  • I'm on the fence on collectResults, although I think maybe renaming it would tip me over. Is there a good name for this that features "all", since this function really checks to make sure all the results were successful? allResults doesn't seem right.
  • I don't really like splitResult. It seems weird that the counts of the returned arrays might be "0, 1" if the input result is a failure. I think I would rather just see this (which could be renamed / cleaned up, but it's the type that matters):
// currently we have this, where the type of people is ([Person], [NSError]) 
let people = splitResult(json["people"].array, Person.createWithJSON)

// do this instead, which makes the type of people Result<([Person], [NSError])>
let people = json["people"].array.map { partitionResults($0.map(Person.createWithJSON)) }

so you know either something failed before you got to the array (people is a .Failure), or you got an array (people is a .Success) and then you partitioned the array into successes/failures. If a project wanted to define something that does what splitResult does, they could, but I don't think it's general enough to be here.


I'll propose one more free function. Currently to implement types, we have to do something like this:

    public static func createWithJSON(value: JSON) -> Result<Person> {
        let name = value["name"].string
        let age = value["age"].int
        let isMarried = value["spouse"].bool

        return name.bind { n in
            age.bind { a in
                isMarried.map { im in
                    return self.init(name: n, age: a, spouse: im)
                }
            }
        }
    }

We could clean that up with a mapAll:

    public static func createWithJSON(value: JSON) -> Result<Person> {
        let name = value["name"].string
        let age = value["age"].int
        let isMarried = value["spouse"].bool

        return mapAll(name, age, isMarried) { (n, a, im) in
            self.init(name: n, age: a, spouse: im)
        }
    }

Swift doesn't have variadic generics, but we could implement for 2, 3, ..., some reasonable upper limit (8? 10?) types. The 3 case is:

func mapAll<T1,T2,T3,U>(r1: Result<T1>, r2: Result<T2>, r3: Result<T3>, f: (T1,T2,T3) -> U) -> Result<U> {
    return r1.bind { s1 in
        r2.bind { s2 in
            r3.map { s3 in
                f(s1, s2, s3)
            }
        }
    }
}

Parser differs in behavior from NSJSONSerialization when a key is repeated in an object

Given the following JSON:

{"a" : 1,"a" : 2}

The NSJSONSerialization class returns the first value 1 for the key a.

The BNR JSONParser returns a value of 2.

I am not suggesting that the BNR version is incorrect, as I believe that the BNR version is more predictable, but it is an instance where the two diverge.

Should we:

  1. Document the difference
  2. Ignore the difference
  3. throw an error if the key is repeated
  4. change the behavior to match Apple's
  5. something else

String memoization

One optimization I discovered in my adventuring with NSJSONSerialization uses a fairly small cache (like ~20) of byte hashes to string instances, thus significantly reducing memory usage for repeated strings in a payload, like enum-style values coming back from a Rails endpoint or execrable usage of the string "true" coming from a Java endpoint.

I experimented with this a bit last week. The optimization rides on a razor-thin edge — i.e., it's really easy to defeat any benefits if you're not careful.

While the current architecture of JSONParser would allow for a simple implementation, it would prevent any potential design changes such as operating on byte streams instead of a single contiguous buffer.

Should be able to use String/Int constants as arguments to JSON accessor functions

Assuming json is a JSON, this works:

let foo = try json.string("foobar")

but this does not:

let myKey = "foobar"
let foo = try json.string(myKey)

Trying to create a .Key (of the PathFragment enum) mysteriously fails with an ambiguity error:

let foo = try json.string(.Key(myKey))

Fully qualifying the path fragment case works but is quite wordy:

let foo = try json.string(JSON.PathFragment.Key(myKey))

How to iterate through a dictionaries values

I have a dictionary like this

[
"identifier": [ "key": "value"],
"identifier": [ "key": "value"],
"identifier": [ "key": "value"],
"identifier": [ "key": "value"]
]

I want to be able to iterate the values of this dictionary so I can parse the sub-dictionaries. How would I do this. So far I've tried this:

let chatSessionList = try value.dictionary("sessions", or: [:])
        chatSessions = try chatSessionList.map {
            _, value in
            return try UserChatSession.init(json: value)
        }

With no luck.

Parser should return JSONResult

Since we don't have a SequenceType implementation now, using collectAllSuccesses and Result<JSON> to unpickle an array of structs is unnecessarily difficult. (Will post a sample once I post a PR for #13.)

Should the `JSONParserType` offer an optional `options` argument

My thought being that as it stands, any configuration of the parser cannot be controlled by the user. For example, if the user wanted to pass the .AllowsFragments argument to the Foundation parser, it isn't possible currently.

My original comment:
Also makes me wonder if the .MutableLeaves or .MutableContainers options make any sense to be "exposed". I am pretty sure the answer is no. But, if we consider some different parser call it XYZParser, let's say that it does have some options which might "tune the parser". Maybe an option for being lenient or something. If this were the case, would it make sense to allow options to be passed to this createJSONFromData function, which would come from the JSON.init(data:usingParser:) function?

Poor deserialization performance

I'm using bnr-swift-json on a project (not linking in case this repo becomes public), and I'm running into an unacceptable performance issue. I'm trying to parse a very large JSON response (~1.5 MB), and it takes nearly 4 seconds (!) on an iPhone 5. Rough estimates from Instruments indicate that the vast majority of that time is being spent in makeJSON, specifically in swift_dynamicCast as it walks through the objects that NSJSONSerialization created and uses casts to figure out their real types.

I can think of 3 possible solutions, although there may be more:

  • Write a pure Swift JSON parser, so we don't have to walk over the object twice (we can create the real type during parsing). I have a rough prototype that's ~7x slower than NSJSONSerialization (but already faster than makeJSON); with some work this is feasible.
  • Make makeJSON lazy somehow - don't do the cast until the caller actually tries to get a real type. This wouldn't actually make it faster if you needed the whole object (probably, although it might since you would only try 1 cast instead of 4), but if you only needed a subset, it would be much faster.
  • Find a fast, open source C or C++ JSON parsing library that we could hook into to create the Swift types during parsing. (I'm not sure how feasible this is, but it might be possible.)

@mdmathias, @zwaldowski, @lyricsboy, thoughts?

JSON, or property lists?

Although the most popular application of this library will most certainly be JSON, it occurs to me that it would be useful for those dealing with other serialization formats that parse in and out of property list objects. Examples include MessagePack, .plists even, etc.

Is there any value in generalizing the library to deal with property list structured data, and provide helpers that deal with the JSON part of things, which is a fairly small part of the problem being solved?

I may be overlooking something about how JSON differs from these other formats that requires the library to be specific to JSON, though.

Wiki page for JSONParser

The page need not be very long or descriptive.

Some basic goals:

  1. Describe why BNR wrote its own parser
  2. Create a performance benchmarking branch that compares using native parser and not.

Add ability to fall-back to defaults for missing values

Sometimes the lack of a key in a JSON payload should be expressed in the API with fallback values, such as a hasFriends key only existing in a payload if it is true on a Person API, otherwise the API contract expects you to assume it's false.

cf. this snippet.

I don't like it as a free function, though. @jgallagher proposed an API on JSONResult:

jResult["myMissingInt"].or(42).int

Documentation audit

Some of the documentation is/will be out-of-date (with respect to #14, #15), we should fix those issues and reach 100% coverage as reported by Jazzy.

Freddy can't parse his first example because it has trailing comma.

  1. See README.md > Usage > Consider some example JSON data:
  2. You can see "people" > "spouse" has trailing comma
  3. So Freddy can't parse it and says 'DictionaryMissingKey' error.

The JSON specification doesn't allow trailing comma, but I think Freddy should allow it like swift.

Parse failures are difficult to debug

Say you're using the Result extensions normally:

let name = myJSON["personName"].string

If this fails for some reason, the resulting error is as follows:

Error Domain=com.bignerdranch.BNRSwiftJSON Code=3 "The operation couldn’t be completed. Unexpected type. `BNRSwiftJSON.JSON` is not convertible to `String`." UserInfo=0x7fd908d145e0 {NSLocalizedFailureReason=Unexpected type. `BNRSwiftJSON.JSON` is not convertible to `String`.}

This is extremely difficult to track down the origin of.

Some suggested solutions in Slack:

  • The subscripts in #50 should capture FILE and LINE and include them in the error, at least in Debug mode.
    • The subscripts in #50 should put the key-path in the error.

Performance benchmarking

Build an idiomatic performance test and benchmark within the unit testing bundle. Ideally, the benchmark would compare against an all-Obj-C solution and both BNRSwiftJSON implementations.

Would it make sense to isolate the code that uses `NSJSONSerialization` into its own file

Currently the JSONParsing.swift file defines the JSONParserType protocol and adds an extension to make JSON conform to it.

There is also an extension to make NSJSONSerialization conform to it.

I would advocate pulling the NSJSONSerialization conformance out into a separate file so that the reliance on Foundation is isolated and doesn't pollute the pure swift awesomeness.

Clean up the README

  • instructions should be more friendly to new people
  • move most of the detailed discussion of implementation into the wiki

Fail early in the Parser if an encoding other than UTF-8 is detected

The parser is documented as not parsing data encoded using anything other than UTF-8. This is fine. I just wonder if it is worth failing immediately if a different encoding is detected. I can only imagine some obscure problem cropping up and causing someone to lose hours because of an encoding issue, and if we could have announced that it is in UTF16 or something it might be helpful.

http://www.ietf.org/rfc/rfc4627.txt

3.  Encoding

   JSON text SHALL be encoded in Unicode.  The default encoding is
   UTF-8.

   Since the first two characters of a JSON text will always be ASCII
   characters [RFC0020], it is possible to determine whether an octet
   stream is UTF-8, UTF-16 (BE or LE), or UTF-32 (BE or LE) by looking
   at the pattern of nulls in the first four octets.

           00 00 00 xx  UTF-32BE
           00 xx 00 xx  UTF-16BE
           xx 00 00 00  UTF-32LE
           xx 00 xx 00  UTF-16LE
           xx xx xx xx  UTF-8

Better error message for bad type.

While doing my comparison with other JSON systems. I was impressed at how much better the error was using Argo when trying to parse an unexpected type. In this example I expected a String but got a Number.

I hope to work on a PR to better express this error and write some tests for it (as part of my pairing with @jeremy-w coming up).

screen shot 2016-01-25 at 2 02 49 pm

screen shot 2016-01-25 at 2 03 06 pm

Split `IgnoringFuture` to a separate type

FutureType.ignoringValue and its implementation are a smell; FutureBoxBase should be a private class. Like EnumerateSequence in the stdlib, IgnoringFuture should be a separate type generic over a FutureType with no type erasure.

Freddy's current behavior when a JSON dictionary has multiple keys of same name is not consistent with other systems.

Currently if a JSON source, which contains a dictionary and that dictionary contains multiple uses of the same key, is parsed with Freddy the second value is returned. In comparison in my testing SwiftyJSON and Argo (which are both based on NSJSONSerialization) the first value is returned.

I would like to work with @jeremy-w on a PR to address this (during our testing pair time) but first I need some guidance on what should the behavior be. In my gut I think we should reject the JSON as invalid, and should they want to accept the faulty JSON tell them to pass in NSJSONSerialization as the parser.

Date parsing?

I didn't see any convenience helpers for date parsing in the library. Would you accept a pull request for that, or is it out of scope?

Large integers cause overflows, especially on 32-bit hosts

Follow-on from #57. Consensus was that JSON.Int should continue to use Swift.Int as its associated type, but that still leaves the issue of overflows at hand. The stdlib asserts inside the parser, generally bringing down the whole app as a result.

Options include:

  • Switch the parser to the &+ et. al. operators and explicitly allow overflows.
  • Switch the parser to the addWithOverflow(_:_:) et. al. functions, catch overflows, and throw errors, thus cancelling the parse.
  • Switch the parser to the addWithOverflow(_:_:) et. al. functions, catch overflows, and return Strings for big integers. This might require some workflow changes in the parser.
  • Switch the parser to the addWithOverflow(_:_:) et. al. functions, catch overflows, and return Doubles for big integers. (undesirable, but I mention it)

Workaround

If this is a problem for you today, you can work around it by using Apple's parser rather than Freddy's via JSON(data:usingParser:) and specifying the parser as NSJSONSerialization.self.

// assuming you have data: NSData
let json = try JSON(data: data, usingParser: NSJSONSerialization.self)

For a bit more discussion around this, see this comment below.

Discussion Summary

Eventual decision was to avoid overflow by treating numbers that would overflow as strings. The value can be pulled out and then parsed from the string if needed by the user of the API. Implemented in #152 and expected to land in Freddy v2.1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.