Code Monkey home page Code Monkey logo

json-patch's Introduction

JSON-Patch

jsonpatch is a library which provides functionality for both applying RFC6902 JSON patches against documents, as well as for calculating & applying RFC7396 JSON merge patches.

GoDoc Build Status Report Card

Get It!

Latest and greatest:

go get -u github.com/evanphx/json-patch/v5

If you need version 4, use go get -u gopkg.in/evanphx/json-patch.v4

(previous versions below v3 are unavailable)

Use It!

Configuration

  • There is a global configuration variable jsonpatch.SupportNegativeIndices. This defaults to true and enables the non-standard practice of allowing negative indices to mean indices starting at the end of an array. This functionality can be disabled by setting jsonpatch.SupportNegativeIndices = false.

  • There is a global configuration variable jsonpatch.AccumulatedCopySizeLimit, which limits the total size increase in bytes caused by "copy" operations in a patch. It defaults to 0, which means there is no limit.

These global variables control the behavior of jsonpatch.Apply.

An alternative to jsonpatch.Apply is jsonpatch.ApplyWithOptions whose behavior is controlled by an options parameter of type *jsonpatch.ApplyOptions.

Structure jsonpatch.ApplyOptions includes the configuration options above and adds two new options: AllowMissingPathOnRemove and EnsurePathExistsOnAdd.

When AllowMissingPathOnRemove is set to true, jsonpatch.ApplyWithOptions will ignore remove operations whose path points to a non-existent location in the JSON document. AllowMissingPathOnRemove defaults to false which will lead to jsonpatch.ApplyWithOptions returning an error when hitting a missing path on remove.

When EnsurePathExistsOnAdd is set to true, jsonpatch.ApplyWithOptions will make sure that add operations produce all the path elements that are missing from the target object.

Use jsonpatch.NewApplyOptions to create an instance of jsonpatch.ApplyOptions whose values are populated from the global configuration variables.

Create and apply a merge patch

Given both an original JSON document and a modified JSON document, you can create a Merge Patch document.

It can describe the changes needed to convert from the original to the modified JSON document.

Once you have a merge patch, you can apply it to other JSON documents using the jsonpatch.MergePatch(document, patch) function.

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	// Let's create a merge patch from these two documents...
	original := []byte(`{"name": "John", "age": 24, "height": 3.21}`)
	target := []byte(`{"name": "Jane", "age": 24}`)

	patch, err := jsonpatch.CreateMergePatch(original, target)
	if err != nil {
		panic(err)
	}

	// Now lets apply the patch against a different JSON document...

	alternative := []byte(`{"name": "Tina", "age": 28, "height": 3.75}`)
	modifiedAlternative, err := jsonpatch.MergePatch(alternative, patch)

	fmt.Printf("patch document:   %s\n", patch)
	fmt.Printf("updated alternative doc: %s\n", modifiedAlternative)
}

When ran, you get the following output:

$ go run main.go
patch document:   {"height":null,"name":"Jane"}
updated alternative doc: {"age":28,"name":"Jane"}

Create and apply a JSON Patch

You can create patch objects using DecodePatch([]byte), which can then be applied against JSON documents.

The following is an example of creating a patch from two operations, and applying it against a JSON document.

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	original := []byte(`{"name": "John", "age": 24, "height": 3.21}`)
	patchJSON := []byte(`[
		{"op": "replace", "path": "/name", "value": "Jane"},
		{"op": "remove", "path": "/height"}
	]`)

	patch, err := jsonpatch.DecodePatch(patchJSON)
	if err != nil {
		panic(err)
	}

	modified, err := patch.Apply(original)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Original document: %s\n", original)
	fmt.Printf("Modified document: %s\n", modified)
}

When ran, you get the following output:

$ go run main.go
Original document: {"name": "John", "age": 24, "height": 3.21}
Modified document: {"age":24,"name":"Jane"}

Comparing JSON documents

Due to potential whitespace and ordering differences, one cannot simply compare JSON strings or byte-arrays directly.

As such, you can instead use jsonpatch.Equal(document1, document2) to determine if two JSON documents are structurally equal. This ignores whitespace differences, and key-value ordering.

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	original := []byte(`{"name": "John", "age": 24, "height": 3.21}`)
	similar := []byte(`
		{
			"age": 24,
			"height": 3.21,
			"name": "John"
		}
	`)
	different := []byte(`{"name": "Jane", "age": 20, "height": 3.37}`)

	if jsonpatch.Equal(original, similar) {
		fmt.Println(`"original" is structurally equal to "similar"`)
	}

	if !jsonpatch.Equal(original, different) {
		fmt.Println(`"original" is _not_ structurally equal to "different"`)
	}
}

When ran, you get the following output:

$ go run main.go
"original" is structurally equal to "similar"
"original" is _not_ structurally equal to "different"

Combine merge patches

Given two JSON merge patch documents, it is possible to combine them into a single merge patch which can describe both set of changes.

The resulting merge patch can be used such that applying it results in a document structurally similar as merging each merge patch to the document in succession.

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	original := []byte(`{"name": "John", "age": 24, "height": 3.21}`)

	nameAndHeight := []byte(`{"height":null,"name":"Jane"}`)
	ageAndEyes := []byte(`{"age":4.23,"eyes":"blue"}`)

	// Let's combine these merge patch documents...
	combinedPatch, err := jsonpatch.MergeMergePatches(nameAndHeight, ageAndEyes)
	if err != nil {
		panic(err)
	}

	// Apply each patch individual against the original document
	withoutCombinedPatch, err := jsonpatch.MergePatch(original, nameAndHeight)
	if err != nil {
		panic(err)
	}

	withoutCombinedPatch, err = jsonpatch.MergePatch(withoutCombinedPatch, ageAndEyes)
	if err != nil {
		panic(err)
	}

	// Apply the combined patch against the original document

	withCombinedPatch, err := jsonpatch.MergePatch(original, combinedPatch)
	if err != nil {
		panic(err)
	}

	// Do both result in the same thing? They should!
	if jsonpatch.Equal(withCombinedPatch, withoutCombinedPatch) {
		fmt.Println("Both JSON documents are structurally the same!")
	}

	fmt.Printf("combined merge patch: %s", combinedPatch)
}

When ran, you get the following output:

$ go run main.go
Both JSON documents are structurally the same!
combined merge patch: {"age":4.23,"eyes":"blue","height":null,"name":"Jane"}

CLI for comparing JSON documents

You can install the commandline program json-patch.

This program can take multiple JSON patch documents as arguments, and fed a JSON document from stdin. It will apply the patch(es) against the document and output the modified doc.

patch.1.json

[
    {"op": "replace", "path": "/name", "value": "Jane"},
    {"op": "remove", "path": "/height"}
]

patch.2.json

[
    {"op": "add", "path": "/address", "value": "123 Main St"},
    {"op": "replace", "path": "/age", "value": "21"}
]

document.json

{
    "name": "John",
    "age": 24,
    "height": 3.21
}

You can then run:

$ go install github.com/evanphx/json-patch/cmd/json-patch
$ cat document.json | json-patch -p patch.1.json -p patch.2.json
{"address":"123 Main St","age":"21","name":"Jane"}

Help It!

Contributions are welcomed! Leave an issue or create a PR.

Before creating a pull request, we'd ask that you make sure tests are passing and that you have added new tests when applicable.

Contributors can run tests using:

go test -cover ./...

Builds for pull requests are tested automatically using GitHub Actions.

json-patch's People

Contributors

ashanbrown avatar bentheelder avatar brettbuddin avatar clagraff avatar cwinters avatar dataturd avatar dependabot[bot] avatar dixudx avatar evanphx avatar howardjohn avatar jimmidyson avatar jkodumal avatar joe2far avatar josieli-google avatar klauspost avatar krishicks avatar liggitt avatar logicalhan avatar lpetrucci-palantir avatar marcelmue avatar mikedanese avatar mikkeloscar avatar mrpotes avatar neo2308 avatar peterbourgon avatar radwaretaltr avatar sbward avatar thajeztah avatar vassilvk avatar zqzten avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

json-patch's Issues

Bug in isArray implementation

The body of isArray is as follows:

    for _, c := range buf {
        switch c {
        case ' ':
        case '\n':
        case '\t':
            continue
        case '[':
            return true
        default:
            break
        }
    }

This seems innocuous enough, but the break is defined to be a break on the switch statement, not the for loop. This means that calls to isArray will always return true if there's a [ anywhere in the JSON document.

Add option for more tolerant "add" which generates missing parts of "path"

Currently (and according to the JSON Patch standard) when performing "add" against a non-existent path, json-patch errors out.

That said, there are a number of use cases where it is significantly more convenient to have a single "add" automatically generate the missing parts of the path, instead of having different patches for different "missing-element" scenarios.

For example, when patching a Kubernetes object which has no annotations in its manifest with the following patch, it would be best if json-patch generated the missing annotations object:

{
  "op": "add",
  "path": "/metadata/annotations/my-annotation",
  "value": "my-value"
}

The suggested solution is to introduce a new ApplyOptions option: EnsurePathExistsOnAdd with default value of false.
When set to true, this option instructs add operations to make sure every element of their path exists, and if it does not, it creates it.

This auto-generation algorithm would be smart enough to recognize the type of container element to create (object or array), based on the value of the subsequent path element.

Additionally, when parts of the path are array indexes that do not exist in their respective arrays, EnsurePathExistsOnAdd pads the containing array with nulls to ensure the array contains enough elements to perform the add operation.

Removing other properties

This library was working fine for the code below, but it's started to remove / delete other properties on an object.

Patching code

func patchUser(user *db.User, patchDetails *[]byte) (*db.User, error) {
    obj, err := jsonpatch.DecodePatch(*patchDetails)
    if err != nil {
        return nil, err
    }

    userJson, err := json.Marshal(user)
    if err != nil {
        return nil, err
    }

    out, err := obj.Apply(userJson)
    if err != nil {
        return nil, err
    }

    tempUser := db.User{}
    err = json.Unmarshal(out, &tempUser)
    if err != nil {
        return nil, err
    }

    return &tempUser, nil
}

User object

{
    "_id" : "57a9c517330eb93e4bd492fa",
    "firstname" : "Charles",
    "lastname" : "Vallance",
    "nickname" : "",
    "email" : "[email protected]",
    "password" : "",
    "timezone" : "",
    "sociallogins" : [ 
        {
            "type" : "Facebook",
            "accesstoken" : "anothersecret",
            "expiry" : "2016-10-08T08:02:03.327Z",
            "refreshtoken" : ""
        }
    ]
}

Patch

[
    {"op":"replace","path":"/timeZone","value":"Pacific/Auckland"},
    {"op":"replace","path":"/nickname","value":"cvallance"}
]

Result

{
    "_id" : "57a9c517330eb93e4bd492fa",
    "firstname" : "",
    "lastname" : "",
    "nickname" : "cvallance",
    "email" : "",
    "password" : "",
    "timezone" : "Pacific/Auckland",
    "sociallogins" : []
}

Expected

{
    "_id" : "57a9c517330eb93e4bd492fa",
    "firstname" : "Charles",
    "lastname" : "Vallance",
    "nickname" : "cvallance",
    "email" : "[email protected]",
    "password" : "",
    "timezone" : "Pacific/Auckland",
    "sociallogins" : [ 
        {
            "type" : "Facebook",
            "accesstoken" : "anothersecret",
            "expiry" : "2016-10-08T08:02:03.327Z",
            "refreshtoken" : ""
        }
    ]
}

Sorry, I don't have time to test it myself right now. I'll can have a look tomorrow night if need be.

Introduce AllowMissingPathOnRemove

This is a proposal for a new package-level flag AllowMissingPathOnRemove.
When this flag is set to true, JSON-Patch remove operations do not fail when the target path is missing from the container being patched.
The value of AllowMissingPathOnRemove defaults to false to maintain backward compatibility with previous functionality.

Reasoning

The current implementation fails the entire patch when a remove operation includes a missing path.
There are use cases where the user just needs an item gone from the final document. If it's already missing from the document, that's ok - don't fail.
An alternative remove-if-it-exists solution based on error handling against the current implementation does not work for multi-operation patches, because if one remove operation fails, the entire patch is failed.

Ordering of object keys is not preserved

As per golang/go#27179, sometimes the changes in some JSON content need to be read and compared by humans, and if the content is reordered by the applying of a JSON patch this is made hard to do.

This library should make the changes specified by patches in a non-destructive way to the existing content to facilitate readability by human users.

Move / replace operations should fail if origin does not exist

According to RFC 6902, a move operation should fail if the from location does not exist. The spec also states that a replace operation should fail if the path does not exist.

My testing shows that neither of these errors are caught at the moment.

Replace op should error out if path doesn't exist

Hi,

As per Json RFC, path must exist for the operation to be successful.
Refer this section from the RFC doc:


   The "replace" operation replaces the value at the target location
   with a new value.  The operation object MUST contain a "value" member
   whose content specifies the replacement value.

   The target location MUST exist for the operation to be successful.

   For example:

   { "op": "replace", "path": "/a/b/c", "value": 42 }

   This operation is functionally identical to a "remove" operation for
   a value, followed immediately by an "add" operation at the same
   location with the replacement value.

But right now if the path doesn't exist, then json-patch doesn't error out and instead add the specified value at the given path.

Eg:

	original := []byte(`{"name": "John", "age": 24}`)
	patchJSON := []byte(`[
		{"op": "replace", "path": "/height", "value": 1.23}
	]`)
	patch, err := jsonpatch.DecodePatch(patchJSON)
	if err != nil {
		panic(err)
	}

	modified, pErr := patch.Apply(original)
	if pErr != nil {
		panic(pErr)
	}
	fmt.Print(modified)
}

output --> {"age":24,"height":1.23,"name":"John"}

Here /height path doesn't exist but replace operation instead of throwing error, adds height field in the json doc.

I'm not sure if this is the required behaviour or error is masked intentionally not to break things if path doesn't exist.

move op for array, the result is not correct

hello,move op should do same thing with remove op and add op, but i found the result is incorrect in some case. original document array has four elements, but after move op, only three elements if from index is the last index. demo is here, please check it, thanks

package main

import (
	"fmt"
	jsonpatch "github.com/evanphx/json-patch"

)

func main() {
	original := []byte(`{"env":[{"name": "weight","value":"121"},{"name": "height","value":"160"},{"name": "age"},{"name": "job","value":"teacher"}]}`)
	patchJSON := []byte(`[
                 {"op": "move", "from":"/env/3","path": "/env/1"}
	]`)

	patch, err := jsonpatch.DecodePatch(patchJSON)
	if err != nil {
		panic(err)
	}

	modified, err := patch.Apply(original)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Original document: %s\n", original)
	fmt.Printf("Modified document: %s\n", modified)
}

Applying Merge Patch

README talks about creating a merge patch, but does this library support applying the merge patch?

p.s. by merge patch I mean {"height":null,"name":"Jane"} and not {"op": "replace", "path": "/name", "value": "Jane"}.

Add line number when error happens

Got following error

Error: strconv.Atoi: parsing "": invalid syntax

when applying the syntax.

- op: add
  path: /spec/template/spec/containers/0/args/
  value: --default-issuer-kind=ClusterIssuer 
- op: add
  path: /spec/template/spec/containers/0/args/
  value: --default-issuer-name=letsencrypt

It would be nice to have the line number when the error happens.

EnsurePathExistsOnAdd fails for missing path components of -1

When EnsurePathExistsOnAdd is set to true and an add operation is performed for a non-existent path which includes -1 in its path components, ApplyWithOptions fails.

This is incorrect, as -1 for a missing path on add should be equivalent to 0.
Making EnsurePathExistsOnAdd treat -1 as 0 for missing paths would be consistent with executing an add operation with -1 against an empty (but existing) array.

Introduce ApplyWithOptions

Implement method func (p Patch) ApplyWithOptions(doc []byte, options *ApplyOptions) ([]byte, error) which implements the functionality of func (p Patch) Apply, but takes its options from parameter options instead of global package-level variables.
ApplyOptions is a struct type which includes the following fields:

  • SupportNegativeIndices bool
  • AccumulatedCopySizeLimit int64
  • AllowMissingPathOnRemove bool

Modify Apply to call ApplyWithOptions and pass the values of package variables SupportNegativeIndices and AccumulatedCopySizeLimit as default options as well as false for option AllowMissingPathOnRemove.

panic instead of returning an error

most of the when pointing to non existing propertis and malformated the patch json cause a panic instead of returning an error.
here are some examples :
this the doc before any patch
{
"creation_date":"2015-09-14T13:21:48.82+04:30",
"host_id":"55f68aa42bdf2711fe000001",
"identifier":"host1",
"key":"host",
"now_playing":null,
"owner":{
"uuid":"1dd60c63-1a49-492b-8298-980d6f1b8e9c",
"username":"ali",
"firstname":"",
"lasttname":""
},
"playlist":{
"playlist_id":"host1",
"songs":{
},
"date":"2015-09-14T13:21:48.82+04:30"
},
"users":[
]
}
these too patches cause panic:
[ { "op": "replace", "path": "/key/2", "value": "host" }]
[ { "op": "replace", "path": "//key", "value": "host" }]

Use semver tags

I'd be nice to have semver tags (e.g. v1.0.0) so I can pin by version rather than hash. And so that I know what to expect when I update.

implement softfail

I'm try to use json-patch to normalize json objects since the input json objects can be very different, e.g. some fields are set, some aren't.

When using op remove or replace json-patch fails and terminates because it failed to apply the patch. I'd like to ignore this kind of error (mapping with a specific key doesn't exist), so all other patches will be applied anyway.

Is there a good way to specify only a subset of operations is enabled?

Is anyone working on a way to specify which operations were supported by which fields of a document. Would that violate the RFC? mholt/binding has this concept of a FieldMap where different options can be specified per field in a data structure.

Would there be any interest in adding something similar to json-patch?

e.g.

type User struct {
Name string
Hobbies []string
}

func (u *User) FieldMap() FieldMap {
return FieldMap {
&u.Name: Field{
Operations: [ Replace ],
},
&u.Hobbies: Field{
Operations: [ Replace, Add, Remove ]
},
}
}

Panic on Apply of op:test with non existing attribute

Hello,

I have tried to apply following patch:

[{"op":"test","path":"/missingAttr"}]

on json:

{"intAttr":10,"strAttr":"string"}

And it resulted in panic:
runtime error: invalid memory address or nil pointer dereference:

With stack:
/usr/local/go/src/runtime/panic.go:679
runtime.panicmem
/usr/local/go/src/runtime/panic.go:199
runtime.sigpanic
/usr/local/go/src/runtime/signal_unix.go:394
github.com/evanphx/json-patch.newLazyNode
.../go/pkg/mod/github.com/evanphx/[email protected]+incompatible/patch.go:61
github.com/evanphx/json-patch.Patch.ApplyIndent
.../go/pkg/mod/github.com/evanphx/[email protected]+incompatible/patch.go:752
github.com/evanphx/json-patch.Patch.Apply
.../go/pkg/mod/github.com/evanphx/[email protected]+incompatible/patch.go:718
cm-backend/server/api.processJsonPatch
....

Expectation was to see an error.

according to RFC6902 JSON patch, replace operation should fail if path not exist before

This issue is the same as the previous issues(#18 and #40).
According to RFC6902 JSON patch, replace operation should fail if path not exist before. From issue#40, the auther may have merged updated code. But now when I compare codes in branch master or tag 4.2.0, I find that the place that should be updated is not updated. Does the author have other concerns? Expect the author to update the code as soon as possible~~

code in branch master and tag 4.2.0 in method replace:

_, ok := con.get(key)
if ok != nil {
	return fmt.Errorf("jsonpatch replace operation does not apply: doc is missing key: %s", path)
}

code may be merged in method replace:

val, ok := con.get(key)
if val == nil || ok != nil {
	return fmt.Errorf("jsonpatch replace operation does not apply: doc is missing key: %s", path)
}

RFC 6902 escaping in merge.go

Hi,

maybe I missed something, but isn't the patch.go about RFC6902 patches while merge.go is about RFC7xxx ?

If so, why do we escape "/" like in RFC 6092 in merge.go?

Thanks

Cannot add to array using negative index.

I am running go version go1.7.3 darwin/amd64 with the latest of evanphx/json-patch.

The bug

Currently, if I try to use the add operation to append to an array, specifying a negative index, I get a runtime panic.

This is what RFC 6902, Section 4.1 has to say about negative indexes during an add operation:

If the "-" character is used to index the end of the array (see RFC6901), this has the effect of appending the value to the array.

The code

Here is the code I used to test:

package main

import (
	"fmt"

	"github.com/evanphx/json-patch"
)

func main() {
	document := []byte(`{"count": 2, "items": ["one", "two"]}`)
	patch := []byte(`[
		{"op": "add", "path": "/count", "value": 3},
		{"op": "add", "path": "/items/-1", "value": "three"}
	]`)

	patchObj, err := jsonpatch.DecodePatch(patch)
	if err != nil {
		panic(err)
	}

	out, err := patchObj.Apply(document)
	if err != nil {
		panic(err)
	}

	fmt.Println(string(out))
}

I expect:

> go run main.go
{"count":3,"items":["one","two","three"]}

but instead I get:

> go run main.go
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0xc5ea0, 0xc42000a110)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/evanphx/json-patch.(*partialArray).add(0xc4200164c0, 0xc42000a558, 0x2, 0xc4200165a0, 0xc42006d008, 0xc42000a501)
	/Users/.../src/github.com/evanphx/json-patch/patch.go:372 +0x2a1
github.com/evanphx/json-patch.Patch.add(0xc420012340, 0x2, 0x4, 0xc42003ddf0, 0xc420016420, 0x0, 0x0)
	/Users/.../src/github.com/evanphx/json-patch/patch.go:425 +0x117
github.com/evanphx/json-patch.Patch.ApplyIndent(0xc420012340, 0x2, 0x4, 0xc420016300, 0x25, 0x30, 0x0, 0x0, 0xd1fe0, 0xc420012300, ...)
	/Users/.../src/github.com/evanphx/json-patch/patch.go:594 +0x484
github.com/evanphx/json-patch.Patch.Apply(0xc420012340, 0x2, 0x4, 0xc420016300, 0x25, 0x30, 0x0, 0x0, 0x63c6c, 0x0, ...)
	/Users/.../src/github.com/evanphx/json-patch/patch.go:570 +0x82
main.main()
	/Users/.../src/github.com/clagraff/main.go:21 +0x106
exit status 2

Additional

I think I have a fix and will submit a PR soon.

That all said, RFC 6901, Section 4 which is referenced in that quoted portion of RFC 6901 I pasted above has this to say about negative array indexes:

Note that the use of the "-" character to index an array will always result in such an error condition because by definition it refers to a nonexistent array element. Thus, applications of JSON Pointer need to specify how that character is to be handled, if it is to be useful.

So maybe this is okay behavior? It is a simple fix and seems to be a nice-to-have, so I will submit a fix for it anyways and let you decide if it seems right.

Please create a newer release that includes #74

Kubernetes now depends on #74 however the latest tagged release, 4.1.0 does not yet have this commit.

The go module system will not choose a commit after the latest tagged release, requiring a replace stanza.

This will cause modules using github.com/evanphx/json-patch and k8s.io/apiserver to require a replace stanza

/cc @caesarxuchao @evanphx

# k8s.io/apiserver/pkg/server
../../../go/pkg/mod/k8s.io/[email protected]/pkg/server/config.go:484:33: undefined: jsonpatch.AccumulatedCopySizeLimit
../../../go/pkg/mod/k8s.io/[email protected]/pkg/server/config.go:488:34: undefined: jsonpatch.AccumulatedCopySizeLimit
FAILED

ref kubernetes/test-infra#12107

Remove may cause invalid index error

Test code as follow

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
	corev1 "k8s.io/api/core/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/serializer"
	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
)

var scheme = runtime.NewScheme()
var codecs = serializer.NewCodecFactory(scheme)

func init() {
	addToScheme(scheme)
}

func addToScheme(scheme *runtime.Scheme) {
	utilruntime.Must(corev1.AddToScheme(scheme))
}

func main() {
	podStr := `
{
  "spec": {
    "tolerations": [
      {
        "key": "role",
        "operator": "Equal",
        "value": "test",
        "effect": "NoSchedule"
      },
      {
        "key": "node.kubernetes.ionot-ready",
        "operator": "Exists",
        "effect": "NoExecute",
        "tolerationSeconds": 300
      },
      {
        "key": "node.kubernetes.io/unreachable",
        "operator": "Exists",
        "effect": "NoExecute",
        "tolerationSeconds": 300
      }
    ]
  }
}
`
	pod := corev1.Pod{}
	deserializer := codecs.UniversalDeserializer()
	if _, _, err := deserializer.Decode([]byte(podStr), nil, &pod); err != nil {
		fmt.Println(err)
		return
	}
	p := `[{"op":"remove","path":"/spec/tolerations/1"},{"op":"remove","path":"/spec/tolerations/2"}]`
	Patch, err := jsonpatch.DecodePatch([]byte(p))
	if err != nil {
		fmt.Println(err)
		return
	}
	bytes, err := Patch.Apply([]byte(podStr))
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(string(bytes))
}

this error happen occur

Unable to access invalid index: 2

patch failed

go version:go1.13.5 windows/amd64

jsonpatch.DecodePatch(patch) ,patch is some data. always return err:

json:cannot unmarshal object into Go value of type jsonpatch.Patch.

It seems golang'json can't deal with jsonpatch.Patch type

Replace does not work for array members

Consider the following JSON:

{ "foo": ["bar"]}

and this patch:

[ { "op": "replace", "path": "/foo/0", "value": "baz"}]

The resulting JSON object should be:

{"foo": ["baz"]}

Currently, this library returns:

{"foo": ["baz","bar"]}

The problem is that the set implementation in partialArray is not actually a set-- it's an append. I've got a proposed PR for this.

extend patch to be able to filter arrays on values

Do you have any interest in extending json-patch to be able to filter on values? e.g. if you have the original json:

{
 "baz": [
    { "bar": "foo"},
    { "bar": "qux"}
  ]
}

and the patch:

 { "op": "replace", "path": "/baz/bar=foo/bar", "value": "baz" } ]

with the patch applied, the json would be:

{
 "baz": [
    { "bar": "baz"},
    { "bar": "qux"}
  ]
}

'go get' fails

Hi,
Tried the 'go get' in the README but I just the response as below:

# go get -u gopkg.in/evanphx/json-patch.v4
go: gopkg.in/evanphx/[email protected]: go.mod has non-....v4 module path "github.com/evanphx/json-patch" at revision v4.7.0

Getting v5 gives a similar error.
What am I doing wrong?
Many thanks

When producing a merge patch, arrays within objects will always be considered different even if they're not

Given this JSON object:

{
  "OuterArray": [
    {
      "InnerArray": [
        {
          "StringAttr": "abc123"
        }
      ],
      "StringAttr": "def456"
    }
  ]
}

If I pass that object as both the original and new object to CreateMergePatch, it will return a non-empty merge patch containing the whole object (although if there were additional top-level values which were identical and not arrays, they would be eliminated correctly)

This is due to lack of handling of arrays being within objects in matchesValue. PR #20 fixes this.

Support use of different JSON library

We heavily use https://godoc.org/github.com/json-iterator/go as an alternative to the standard encoding/json library, because of capabilities it provides that the standard library does not (e.g. sorting of map keys), as well as performance. We would like to be able to configure json-patch to use a different JSON library for Marshal() and Unmarshal() calls.

If json-patch defined a public interface that supported all of the functions on the standard encoding/json library, and supported the ability to configure the json library to use, then this would enable us to use json-patch but have json-patch delegate the marshal / unmarshal / indent / etc. calls to our desired JSON library.

invalid v > 1 go module

Since this is tagged as v4.X and has a go.mod, the import path must be github.com/evanphx/json-patch/v4 including in the module directive and in any code importing packages across the repo and externally.

https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Trying to import v4.6.0 from a project using go modules does not work without this, v4.5.0+incompatible is the last version that works with go modules (this works because it is "incompatible" but not opted into modules by not having a go.mod).

Removing go.mod / go.sum would also work, v4.7.0 could be imported as +incompatible with no other changes.

I'm not sure which route you'd prefer, but currently projects using go modules are stuck on v4.5.0. I'd be happy to send a PR for whatever option is acceptable.

Apply JSON Patch failed

Hi, all

I want to use JSON Patch to add a name is injector.test.io/status key into the json document, but it failed.

error logs:

jsonpatch add operation does not apply: doc is missing path: "/annotations/injector.test.io/status"

But i rename this key is injector.test.io.status, it works well.

I want to know how to do if want to add similar to xx/yy key into json document? Thanks.

Below is my demo code:

package main

import (
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	original := []byte(`{"annotations": {"injector.test.io/inject": "true"}}`)
	patchJSON := []byte(`[
		{"op": "add", "path": "/annotations/injector.test.io/status", "value": "true"}
	]`)

	patch, err := jsonpatch.DecodePatch(patchJSON)
	if err != nil {
		panic(err)
	}

	modified, err := patch.Apply(original)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Original document: %s\n", original)
	fmt.Printf("Modified document: %s\n", modified)
}

License

Please specify a source code license, if you intend for others to be able to use your code.

add op for a patch does not work as expected for adding new member

According to https://tools.ietf.org/html/rfc6902#page-12:

   An example target JSON document:
   { "foo": "bar"}
   A JSON Patch document:
   [
     { "op": "add", "path": "/baz", "value": "qux" }
   ]
   The resulting JSON document:
   {
     "baz": "qux",
     "foo": "bar"
   }

It shows if there is no key baz, it will be created for add op.

However, it seems not work with json-patch. I'm working on kubefed project, I use below patch

     - path: "/metadata/annotations/foo"
       op: "add"
       value: "bar"

The add result shows add operation does not apply: doc is missing path: "/metadata/annotations/foo": missing value

I checked code

if con == nil {

It seems before add, the code needs to get it first. I guess this is a bug. For add op, we should try the prefix if the whole path cannot be find.

Any comments?
@evanphx

Create Merge/Patch

Would it make sense to have a function that can create a merge/patch by comparing a base/changed document. I would imagine it could be done rather easily with the existing code.

Creating RFC6902 Patch Documents

Hi,
Is there a plan to create the RFC6902 patches as well as applying them? There are a few libraries floating around and this looks to be the most mature so having a canonical implementation of both 6902 and 7396 would be great.

[
  {"op": "replace", "path": "/name", "value": "Jane"},
  {"op": "remove", "path": "/height"}
]

Paths without a leading `/` are silently accepted

The JSONPatch library splits the path by / and discards the first segment, assuming it is empty:

json-patch/patch.go

Lines 338 to 344 in cb8f3b5

split := strings.Split(path, "/")
if len(split) < 2 {
return nil, ""
}
parts := split[1 : len(split)-1]

This means that an operation with a patch like bogus/path/to/item will successfully apply to /path/to/item

Seen in kubernetes/kubernetes#81422

Before fixing this, we should consider the compatibility implications of rejecting patches that were previously accepted.

jsonpatch

hi,

i have two json struct like the following, which are equal (only the order of keys is shifted). why is jsonpatch.Equal not returning true?

s1 := []byte(`"SelectedData":[{"ID": 5, "bla": "hello5"},{"ID": 1, "bla": "hello1"},{"ID": 2, "bla": "hello2"}]`)
s2 := []byte(`"SelectedData":[{"ID": 1, "bla": "hello1"},{"ID": 5, "bla": "hello5"},{"ID": 2, "bla": "hello2"}]`)
fmt.Println("jsonpatch.Equal:", jsonpatch.Equal(s1, s2))

if not how can I make it work?

Equal function segfaults if JSON value is null

When comparing objects using jsonpatch.Equal, comparisons of objects recurse to compare values. However, if either value is null (but not both), the code tries to dereference null and causes a segmentation fault.

The following code triggers the bug in v4.5.0, the issue seems to be that the recursive call to equal should first check if the one of the value is nil. See patch.go line 216.

package main

import (
	"encoding/json"
	"fmt"

	jsonpatch "github.com/evanphx/json-patch"
)

func main() {
	a := map[string]interface{}{
		"key1": "value",
		"key2": "other",
	}

	b := map[string]interface{}{
		"key1": "value",
		"key2": nil,
	}

	jsonA, _ := json.Marshal(a)
	jsonB, _ := json.Marshal(b)

	fmt.Printf("A: %s\nB: %s\n", string(jsonA), string(jsonB))

	fmt.Printf("A = B? %v\n", jsonpatch.Equal(jsonA, jsonB))
	fmt.Printf("B = A? %v\n", jsonpatch.Equal(jsonB, jsonA))
}

Replace for PATCH doesn't seem to be working as per rfc6902

I am trying to use replace and as per rfc6902 The target location MUST exist for the operation to be successful.
When I do that:

var payload = []byte(`[{"op":"replace", "path":"/first_name", "value":"Bond, James Bond"}]`)
var doc = []byte(`{"firstName":"John Doe"}`)

func main() {
	obj, err := jsonpatch.DecodePatch(payload)
	if err != nil {
		println(err.Error())
	}

	out, err := obj.Apply(doc)
	if err != nil {
		println(err.Error())
	}

	println(string(out))
}

I am getting

{"firstName":"John Doe","first_name":"Bond, James Bond"}

where I expected it to fail because doc do not have /first_name

If anyone can confirm it is a bug, I can open a PR

Test Entire Document

Does this library support testing an entire JSON document by specifying a root "/" path? I see that it works for replace. If we can't currently test against root "/", is that something folks would be interested in me opening a PR for?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.