I'd like to be able to easily catch bad paths (e.g., due to a typo or misconfiguration):

       paths := []string{
		"Person.FirstNaaaame",  // bad
		"Person.LastName",
	}
	// create mask
	mask, err := fu.MaskFromPaths(paths, namer)
        // desire
        err = fu.Validate(mask, someStruct}

Is there an existing way to validate that all paths refer to an actual field? I'm asking before I implement something to do this.

When building a project following the examples, I receive the following notice:
google.golang.org/protobuf/protoc-gen-go/generator: cannot find module providing package google.golang.org/protobuf/protoc-gen-go/generator

Unfortunately, the new package does not export their GoCamelCase func.

Not sure how you prefer to address this, but I do think the docs should be updated to an alternative, if one exists.

is it possible to support protobuf wrapper values for get the value directly.for now feildmask-util lib,will get a {"value":somevalue}

com_github_mennanov_fieldmask_utils/copy.go:78:28: call of reflect.ValueOf copies lock value: google.golang.org/protobuf/types/known/anypb.Any contains google.golang.org/protobuf/internal/impl.MessageState contains sync.Mutex

Ref: https://stackoverflow.com/questions/64183794/why-do-the-go-generated-protobuf-files-contain-mutex-locks

Sorry I don't know what other sort of context you need here, I'm getting this error in a build for some other service at my org when updating another dependency that then requires me to update bazel_rules_go. It's a mess.

If I can provide any more context please let me know. Although the error is happening because I'm upgrading some bazel package, the error looks to be a gRPC violation that became more strict in later versions.

I'm trying to validate/limit writes to a proto message that includes a oneof group but can't work out how to combine MaskFromProtoFieldMask and StructToStruct.

The issue I'm having is that fieldmaskpb.New correctly validates the oneof fields as if they were top level fields, but fieldmask-utils expects them to be nested inside the oneof group name.

Example

Given a proto like this

message Msg {
  oneof name {
    string formal = 1;
    string casual = 2;
  }
}

This does not return an error: fieldmaskpb.New(&pb.Msg{}, "formal") but passing "name" or "name.formal" does. On the util side, using fieldMaskUtils.MaskFromProtoFieldMask(fields, strcase.ToCamel) however results in a Mask that doesn't work with that proto.

Hope that all makes sense?

I've also submitted a ticket on the proto side to hopefully make this easier: golang/protobuf#1344

Hi,

I'm wondering if its possible to apply a field mask and map to a destination struct where all fields are pointers and any fields excluded after applying the mask are set to nil - as shown in the code snippet below:

package main

import (
  "fmt"

  fmutils "github.com/mennanov/fieldmask-utils"
)


func main() {
	type Source struct {
		FieldOne string
		FieldTwo int
	}

	type Dest struct {
		FieldOne *string
		FieldTwo *int
	}

	source := Source{FieldOne: "hello", FieldTwo: 5}
	dest := new(Dest)

	mask := fmutils.MaskFromString("FieldTwo")
	err := fmutils.StructToStruct(mask, source, &dest)
	fmt.Println("error", err)
	fmt.Println(dest)
}

However, I encounter the following error: error src <int Value>, int is not addressable

My use case here is that I'm trying to implement an gRPC Update endpoint that supports partial updates and I'm using a SQL query builder (squirrel) so I need a way that I can dynamically construct the UPDATE query and only include fields that were contained in the update mask. So in the example above, it would be any non nil fields on the Dest struct.

The reason I don't just use an identical struct (no pointers), is because one wouldn't be able to distinguish between go zero values and values that a user actual explicitly provided in their update request (e.g. if they wanted to set an integer field to zero).

I did think about using the StructToMap method but then there is also some challenges with mapping between the field names in my proto message and the column names in my DB.

I also tried and saw your other repo https://github.com/mennanov/fmutils and previously I had pretty much the same pattern you had in this test but I didn't like the fact I had to do an extra request to fetch the existing resource in the database for every update request.

Do you know of any elegant ways to accomplish dynamic UPDATE query generation? I'd love to hear any thoughts you might have on this!

Also a little side question: in fmutils repo I can simply do

fmutils.Filter(protoMessage, req.Msg.UpdateMask.Paths) and everything works correctly, whereas in this repo it seems that one must apply camel casing first to the proto FieldMask?

Thanks!

time.Time transport to empty map by structx.StructToMapByFieldMask

println(fmt.Sprintf("%T", updateField["CreateTime"]))----------> map[string]interface {}

	type A struct {
		B string
	}
	a := &A{
		B: "b",
	}
	mask, _ := fm.MaskFromPaths([]string{}, generator.CamelCase)
	dest := &A{}
	_ = fm.StructToStruct(mask, a, dest)

	fmt.Println(dest)
	// got: &{b}
	// expect: &{}

I am confused with the copy result. When the mask (paths) is empty, what I expect is nothing will be copied. But the result is everything is copied.

I am wondering what's the purpose to copy everything when the mask is empty?

Why is structToMap supported but not mapToStruct?

Thank you.

type MomentInfo struct {}
type list struct {
    MomentInfos []*MomentInfo `json:"moment_infos"`
}

old := &list{[]*MomentInfo{}}
new := map[string]interface{}

_ = fieldmask_utils.StructToMap(mask, old, new, fieldmask_utils.WithTag(`json`))

jsonStr, _ := json.Marshal(new)
t.Log(new, "\n", string(jsonStr))

print

map[moment_infos:[]] 
{"moment_infos":null}

why after json.Marshal got a nil pointer but not an empty slice?

golang 1.18
fieldmask-utils 0.5.0

Context

I'm trying to convert a proto message to map using this library in order to perform a PATCH update in an endpoint. I'm getting an empty map instead of a single field map with the respective value.

Relevant code

mask := &fieldmaskpb.FieldMask{Paths: []string{"title"}}
mask.Normalize()
task := &tasksv1.Task{
  Title: "test",
}
if !mask.IsValid(task) {
  return nil, errors.New("invalid mask")
}
protoMask, err := fieldmask_utils.MaskFromProtoFieldMask(mask, func(s string) string { return s })
if err != nil {
  return nil, err
}
m := make(map[string]any)
if err = fieldmask_utils.StructToMap(protoMask, task, m); err != nil {
  return nil, err
}
fmt.Println("Map:", m)

Current behavior

m is empty

Expected behavior

m := map[string]any{
  "title": "test",
}

Reference

Protobuf messages: https://github.com/marcoshuck/todo/blob/main/api/tasks/v1/tasks.proto#L60

https://github.com/mennanov/fieldmask-utils/blob/f9573a9a16fd68cd4ab04fca7d1b16dadc7e346a/copy.go#L74C4-L74C14

now mask.Filter use the FiledName of object, However, path also input json_tag, I hope this package can add SrcTag Option for this scene

Hi, thanks for the lib.

Currently, when copying structs, the behavior of preserving fields in dst not consistent. Top-level struct fields are always preserved, but nested struct fields are always zeroed (if mask targets other fields).

Example test case:

func TestStructToStruct(t *testing.T) {
	type (
		Author struct {
			FirstName string
			LastName string
		}
		Document struct {
			Title string
			Date string
			Author Author
		}
	)

	dst := &Document{
		Title:  "Hello, World!",
		Date: "2019-05-01",
		Author: Author{
			FirstName: "John",
			LastName:  "Doe",
		},
	}
	src := &Document{
		Title: "Hello, updated World!",
		Author: Author{
			FirstName: "Johnny",
		},
	}
	expected := &Document{
		Title:  "Hello, updated World!",
		Date: "2019-05-01",
		Author: Author{
			FirstName: "Johnny",
			LastName:  "Doe",
		},
	}

	mask := fieldmask_utils.MaskFromString("Title,Author{FirstName}")
	err := fieldmask_utils.StructToStruct(mask, src, dst)
	require.NoError(t, err)

	assert.Equal(t, expected, dst)
}

In the result of this example, field "Date" is preserved in dst, but field "Author.LastName" is not.

I don't believe StructToStruct is handling anypb correctly. Take a look at the attached patch:
any_existing.patch.txt

It appears to be copying other fields. Notice the ExtraUser Role field has changed even though it isn't included in the mask.

Edit: Also, the ExtraUser Name field should be unchanged (since it isn't included in the mask), but is changing to "username".

*int64 field got panic by StructToMap()，stack as below

example:

import (
	msku "github.com/mennanov/fieldmask-utils"
)

type example struct {
	P2Int64  *int64
	WhatEver string
}

func willPanic() {
	var (
		i         int64 = 666
		updateMap       = make(map[string]interface{})
	)

	mask, _ := msku.MaskFromPaths(
		[]string{"P2Int64", "WhatEver"},
		func(s string) string { return s },
	)

	// panic here
	_ = msku.StructToMap(
		mask,
		&example{
			P2Int64:  &i,
			WhatEver: "what ever",
		},
		updateMap,
	)
}

some code may help:

mask, _ := fieldmask_utils.MaskFromProtoFieldMask(GetFieldMask(), generator.CamelCase)

fieldmask_utils.StructToStruct(fieldmask_utils.MaskInverse(mask), src, dst)

FROM readme example:

mask := fieldmask_utils.MaskInverse{"Id": nil, "Friends": fieldmask_utils.MaskInverse{"Username": nil}}

can also write as

fieldmask_utils.Mask{"Id": nil, "Friends": fieldmask_utils.MaskInverse{"Username": nil}}

We are using fieldmask-utils here at the New York Times for a proof of concept - it's working well so far, except that it required us to bump up to v1.4.2 of protobuf from v1.3.5. This is causing a big performance issue in our use case (marshaling is almost 2-3x slower). My question is whether fieldmask-utils can work with version 1.3.5? Does it rely on anything specific in v1.4.2?

@mennanov nil slice is being set as empty

 func TestStructToStruct_EntireSlice_NonEmptyDst(t *testing.T) {
	type A struct {
		Field1 string
		Field2 int
	}
	type B struct {
		Field1 string
		A      []A
	}
	src := &B{
		Field1: "src StringerB field1",
		A:      nil,
	}
	dst := &B{
		Field1: "dst StringerB field1",
		A: []A{
			{
				Field1: "dst StringerA field1 0",
				Field2: 10,
			},
			{
				Field1: "dst StringerA field1 1",
				Field2: 20,
			},
			{
				Field1: "dst StringerA field1 2",
				Field2: 30,
			},
		},
	}

	mask := fieldmask_utils.MaskFromString("Field1,A")
	err := fieldmask_utils.StructToStruct(mask, src, dst)
	require.NoError(t, err)
	assert.Equal(t, &B{
		Field1: src.Field1,
		A:      src.A,
	}, dst)
}

output -
@@ -2,3 +2,4 @@ Field1: (string) (len=20) "src StringerB field1", - A: ([]fieldmask_utils_test.A) <nil> + A: ([]fieldmask_utils_test.A) { + } })