Code Monkey home page Code Monkey logo

jsondiff's Introduction

JsonDiff library

The main purpose of the library is integration into tests which use json and providing human-readable output of test results.

The lib can compare two json items and return a detailed report of the comparison.

At the moment it can detect a couple of types of differences:

  • FullMatch - means items are identical.
  • SupersetMatch - means first item is a superset of a second item.
  • NoMatch - means objects are different.

Being a superset means that every object and array which don't match completely in a second item must be a subset of a first item. For example:

{"a": 1, "b": 2, "c": 3}

Is a superset of (or second item is a subset of a first one):

{"a": 1, "c": 3}

Library API documentation can be found on godoc.org: https://godoc.org/github.com/nsf/jsondiff

You can try LIVE version here (compiled to wasm): https://nosmileface.dev/jsondiff

The library is inspired by http://tlrobinson.net/projects/javascript-fun/jsondiff/

jsondiff's People

Contributors

dferstay avatar juanefec avatar nsf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jsondiff's Issues

Compare options are not optional

Trying to compare identical empty objects or other simple objects results in a panic if no options are provided. The name is a little misleading since options imply (to me at least) that they are not required.

For instance:
diff, _ := jsondiff.Compare([]byte({}), []byte({}), nil)

Adding opts := DefaultConsoleOptions() and passing it in (as in the unit tests) prevent it from panicking. I didn't get a chance to see what it is expecting just yet.

Output:

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5b272a]

goroutine 6 [running]:
testing.tRunner.func1.1(0x5d7ce0, 0x760ed0)
	/usr/local/go-faketime/src/testing/testing.go:1076 +0x30d
testing.tRunner.func1(0xc000001b00)
	/usr/local/go-faketime/src/testing/testing.go:1079 +0x41a
panic(0x5d7ce0, 0x760ed0)
	/usr/local/go-faketime/src/runtime/panic.go:969 +0x175
github.com/nsf/jsondiff.(*context).printDiff(0xc000045ec0, 0x5d6c60, 0xc000073260, 0x5d6c60, 0xc000073290)
	/tmp/gopath249660858/pkg/mod/github.com/nsf/[email protected]/jsondiff.go:329 +0x4ea
github.com/nsf/jsondiff.Compare(0xc00002c9b0, 0x8, 0x8, 0xc00002c9b8, 0x8, 0x8, 0x0, 0x4b5a86, 0x4af9f070, 0x0)
	/tmp/gopath249660858/pkg/mod/github.com/nsf/[email protected]/jsondiff.go:414 +0x296
main.Test_jsondiffnoopts(0xc000001b00)
	/tmp/sandbox175228423/prog.go:11 +0xaa
testing.tRunner(0xc000001b00, 0x61efb8)
	/usr/local/go-faketime/src/testing/testing.go:1127 +0xef
created by testing.(*T).Run
	/usr/local/go-faketime/src/testing/testing.go:1178 +0x386

Simple examples here: https://play.golang.org/p/vSdHmaOY3zw

On a different note, thank you for sharing this repo. I have a project where we are not using it for debugging, but it is useful for finding superset matches in arbitrary json bodies.

Providing nil options causes a panic

The Compare method takes a parameter of *Options. However, providing a value of nil here causes a panic where it could work correctly.

This code works fine:

		options := jsondiff.DefaultJSONOptions()
		diff, _ := jsondiff.Compare(body, []byte(expected), &options)

Whereas this panics as below:

		diff, _ := jsondiff.Compare(body, []byte(expected), nil)
2021/06/02 07:55:55 http: panic serving 127.0.0.1:53003: runtime error: invalid memory address or nil pointer dereference
goroutine 11 [running]:
net/http.(*conn).serve.func1(0xc00012cbe0)
	/Users/coxg/.gvm/gos/go1.16.4/src/net/http/server.go:1824 +0x153
panic(0x131a260, 0x1582ed0)
	/Users/coxg/.gvm/gos/go1.16.4/src/runtime/panic.go:971 +0x499
github.com/nsf/jsondiff.(*context).printDiff(0xc000117960, 0x1318660, 0xc00007f470, 0x1318660, 0xc00007f4d0)
	/Users/coxg/.gvm/pkgsets/go1.16.4/global/pkg/mod/github.com/nsf/[email protected]/jsondiff.go:340 +0x4ea
github.com/nsf/jsondiff.Compare(0xc000172000, 0x21, 0x200, 0xc000022390, 0x21, 0x30, 0x0, 0xc00005ba30, 0x1284b1f, 0x1346e60)
	/Users/coxg/.gvm/pkgsets/go1.16.4/global/pkg/mod/github.com/nsf/[email protected]/jsondiff.go:425 +0x296
github.com/sazzer/gomockserver.MatchJSON.func1(0xc000168200, 0xc00005ba01)
	/Users/coxg/source/me/gomockserver/json.go:32 +0x198
github.com/sazzer/gomockserver.MatchRuleFunc.Matches(0xc0000681a0, 0xc000168200, 0xc00005bb40)
	/Users/coxg/source/me/gomockserver/matchers.go:16 +0x30
github.com/sazzer/gomockserver.MatchRules.Matches(0xc00005ef80, 0x1, 0x1, 0xc000168200, 0x0)
	/Users/coxg/source/me/gomockserver/matchers.go:25 +0x68
github.com/sazzer/gomockserver.(*Match).Matches(...)
	/Users/coxg/source/me/gomockserver/match.go:13
github.com/sazzer/gomockserver.(*handler).ServeHTTP(0xc00000e198, 0x13e11d0, 0xc0001640e0, 0xc000168200)
	/Users/coxg/source/me/gomockserver/handler.go:13 +0xa5
net/http.serverHandler.ServeHTTP(0xc000164000, 0x13e11d0, 0xc0001640e0, 0xc000168200)
	/Users/coxg/.gvm/gos/go1.16.4/src/net/http/server.go:2887 +0xa3
net/http.(*conn).serve(0xc00012cbe0, 0x13e1800, 0xc00002e980)
	/Users/coxg/.gvm/gos/go1.16.4/src/net/http/server.go:1952 +0x8cd
created by net/http.(*Server).Serve
	/Users/coxg/.gvm/gos/go1.16.4/src/net/http/server.go:3013 +0x39b

Cheers

Allow use of an epsilon when comparing numeric values

We'd like to use jsondiff for comparing JSON that is output from some computation to a static "expected" JSON object. This raises the problem of how to handle floating point numbers. For example, if you run 30.33 - 29 - 0.33 you may get a result something like 0.9999999999999982, which we would like to consider the same as an "expected" value of 1.0.

The current behavior is for jsondiff to compare the original string representations of numeric values. This may be a reasonable default behavior for a generic json library like this, and is certainly the "right" method for some use cases. Additionally, comparisons using an epsilon are impossible to generalize for all use cases, since the optimal epsilon value may depend on the relative scale of the numbers that were used to produce the final result (e.g. 2.3 - 1.2 vs 2.3e10 - 2.29999999988e10). So I think it makes sense to make the use of floats optional.

A related issue with using the string representation for comparisons is that the exact same numeric value can be represented in different ways. For example, the current behavior considers 1 and 1.0 to be different. Same goes for 1e10 and 10e2. So the proposal is for these cases to also be considered the same when using floats.

Add a new function that accepts an interface instead of byte slice to diff against

I'm using github.com/yalp/jsonpath in a project used for testing my apis. It allows me to just look at the subset of the response payload that i'm interested in for the relevant test case. That package returns an interface{} for the value at the specified path instead of a []byte. I have a use-case where I'd like to perform a superset match on a map in the payload.

For example, my api returns something like:

{
    "settings": {
        "localization": {
            "language": "foo"
        },
        "meta": {
            "lastModified": "bar",
            "displayName": "baz",
            "file": {
                "size": "bar",
                "type": "baz"
            }
        }
    }
}

In my test I want to verify some of the values under settings.meta. specifically the displayName string and file map. yalp/jsonpath returns me an interface{} which i can then cast to map[string]interface{} for everything under settings.meta. If I then want to jsondiff.Compare it I have to json.Marshal it back to a []byte.

Today, jsondiff.Compare immediately decodes the []byte into interface{} objects. Why not introduce a new public function that just takes the first parameter as an interface{} to avoid needing to re-marshal before the call? Then the assertion could look like:
jsondiff.CompareInterface(metaInterface, []byte(`{"displayName":"baz","file":{"size":"bar","type":"baz"}}`))

DefaultJSONOptions produces invalid json as diff string

When using the json options, the returned diff string produces invalid JSON preventing it to be unmarshaled into something more usable.

Example

opts := jsondiff.DefaultJSONOptions()

diff, diffStr = jsondiff.Compare(j0, j1, &opts)
if diff != jsondiff.FullMatch {
   fmt.Println(diffStr)

   m := make(map[string]interface{})
   err := json.Unmarshal([]byte(diffStr), m)
    if err != nil {
       log.Fatal(err)
}

Produces

{
    "prop-removed":{"error": {}
        "prop-removed":{"code": -12001,}
        "prop-removed":{"message": "rate limited"}
    "prop-removed":{}},
    "result": {"changed":[null, "otherVal"]}
}
FATA[2022-10-13T12:05:16-07:00] invalid character '"' after object key:value pair

Removing item from top of an array yields noisy output

Given an array {"example":["a", "b", "c", "d"]}, removing the top item a results in a diff with three change lines and one delete for the last item, d. I would expect this to instead result in a single delete for a and no line changes.

Wasm demo ?

Any chance you can upload to google or explain the wasm demo ?

I would like to extend it

Difference is invalid/misleading when comparing a 'null' JSON value and a nested JSON value (array / object)

Toy code:

Run in playground: https://go.dev/play/p/XGfPuf_Px8a

	j := jsondiff.DefaultJSONOptions()
	a := `{"foo":null}`
	b := `{"foo":["bar"]}`

	_, diff := jsondiff.Compare([]byte(a), []byte(b), &j)
	fmt.Println("comparing null to array is broken")
	fmt.Println(diff)

	a = `{"foo":null}`
	b = `{"foo":{"value": "bar"}}`

	_, diff = jsondiff.Compare([]byte(a), []byte(b), &j)
	fmt.Println("comparing null to object is broken")
	fmt.Println(diff)

	a = `{"foo":null}`
	b = `{"foo":"bar"}`

	_, diff = jsondiff.Compare([]byte(a), []byte(b), &j)
	fmt.Println("comparing null to a non-nested value works!")
	fmt.Println(diff)
comparing null to array is broken
{
    "foo": {"changed":[null, []]}
}
comparing null to object is broken
{
    "foo": {"changed":[null, {}]}
}
comparing null to a non-nested value works!
{
    "foo": {"changed":[null, "bar"]}
}

If a field has a json 'null' value in either of the JSON objects, and the other value is a nested value (array or object), the comparison fails to print out the correct value for the other object field's non-null value. See the playground example above for an illustration.

One way to hack a fix is convert all nil slice values to empty slice values in the struct before serializing to json.

On second glance, it looks like the more general issue is with comparing any JSON values of different type: https://go.dev/play/p/fo0aUPTuAB8.

It would be nice if in this case, we emitted the string-fied value of the types, instead of the zero type.

Windows color support using DefaultConsoleOptions

Thank you for this library. I am a Windows user.

I would like to improve the display of special characters with color on Windows. when I install this library in a Windows environment and use jsondiff.DefaultConsoleOptions() to display the diff, if the diff is present, it looks like this.

        , want=[
          {
            "user_id": "001",
            "name": "gopher"
          },
          {
            "user_id": "002",
            "name": "rubyist"
          }
        ]
        , diff=[
            {
                "name": ?[0;33m"rubyist" => "gopher"?[0m,
                "user_id": ?[0;33m"002" => "001"?[0m
            },
            {
                "name": "rubyist",
                "user_id": "002"
            }
        ]

Strings like ?[0;33m and ?[0m are hard to see.

Array type attribute comparisons does not return valid results

Example:

  lastMetadata := map[string]interface{}{
		  "users": []interface{}{},
	  }
  newMetadata := map[string]interface{}{
	  "users": []interface{}{
		  map[string]interface{}{
			  "foo": "bar",
		  },
	  },
  }
  lastMetadataByte, _ := json.Marshal(lastMetadata)
  newMetadataByte, _ := json.Marshal(newMetadata)
  opt := jsondiff.DefaultJSONOptions()
  opt.SkipMatches = true
  _, resultStr := jsondiff.Compare(lastMetadataByte, newMetadataByte, &opt)

resultStr content (not valid JSON):
{
"users": [
"prop-added":{{}
"prop-added":{"foo": "bar"}
"prop-added":{}}
]
}

Print all items in array

I have created a fork but since there is minimal comments, it's hard to work out what's going on.

@nsf If I want all array items printed (even if it's a SupersetMatch and the arrays should not be printed), where is the general area that I should change.

i.e. Say there is an array of items but only first item exists, currently that is considered a SupersetMatch and the other items are not displayed. How can I display them for my fork?

Support displaying only areas of diff

First of all, great library! This is a tricky problem to solve and it's done very nicely in jsondiff!

Now, I am diffing very large JSON files, and I would like to only output the actual differences, not the entire doc if possible.

I couldn't see an existing way to do this, but if there is, please let me know!

Either a config setting that would allow you to display X lines/objects above and below each diff, or potentially a way to roll up previous fields?

The roll-up would ideally look something like this:
From online demo:

{
    "content": [
        {
            "align": "center",
            "content": [
                {
                    "content": "And please, feel free to send us your feedback and comments to ",
                    "style": {
                        "bold": 2,
                        "color": "FFFFFF",
                      - "fontFamily": "Arial",
                        "italic": 2,
                      + "name": "Arial",
                        "size": 20,
                      ~ "underline": 1 => 2
                    }
                },
...

Rolled up:

{
    "content": [
        {
            "content": [
                {
                    "style": {
                        "bold": 2,
                        "color": "FFFFFF",
                      - "fontFamily": "Arial",
                        "italic": 2,
                      + "name": "Arial",
                        "size": 20,
                      ~ "underline": 1 => 2
                    }
                },
...

Or potentially by separating parent objects using a dot notation:

content.content.style {
                        "bold": 2,
                        "color": "FFFFFF",
                      - "fontFamily": "Arial",
                        "italic": 2,
                      + "name": "Arial",
                        "size": 20,
                      ~ "underline": 1 => 2
                    }

It's a tricky one, but to start with, just having an option to say, only display X lines above and below the change, might be enough!

Clarify License

I'd like to use this library for one of my projects. Can you clarify which license applies? Is it Apache 2 or something else?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.