infobloxopen / atlas-app-toolkit Goto Github PK

The size parameter of the randBytes function is confusing, as this function returns half the size specified. If I specify a size of 10, I would expect 10 bytes back. And if size is odd I get less than half.

ctx := trace.NewContext(..)
span, _ := trace.FromContext(ctx)

leads to an empty context, fix issue seen in these unit tests 3047632

Currently, you reference the collection operators middleware like this:

mw.WithCollectionOperator()

We should instead change WithCollectionOperator to match grpc-ecosystem's naming convention:

collectionoperators. UnaryServerInterceptor

or something like that.

Using option

mux := runtime.NewServeMux(append(opts, runtime.WithMarshalerOption("*", &runtime.JSONPb{OrigName: true, EmitDefaults: true}))...)

so that default values are emitted instead of omitted causes a panic in the contacts app, if a WKT is added to the .proto schema but is not set in requests.

Cause appears to have something to do with the gateway/response.go unmarshalling the response into a map, adding the 'success' response section and then remarshalling.

• need to parameterize the swagger location in the Config struct instead of hard-coding it
• need to set sensible defaults for development to file
• need to ensure the docker build imports the swagger file

I'm curious what the use cases for this client interceptor are and if they work. The way it's implemented, it doesn't seem to do anything useful. It adds fields like account_id to the logger sourced from the incoming metadata, while clients utilize outgoing metadata. This is probably a bug. It seems like it would be more valuable to log the subject that is being used in the client, not the subject used in whatever call triggered it (which is only set for service-to-service cases, like the test exercises). In the service-to-service case, I would expect these fields to already be set in the logger by the server interceptor, meaning the client interceptor isn't adding anything beyond what the logrus client interceptor already does.

HI, I receive this error when calling my api and trying to filter
for example http://localhost:7778/api/clients?filter=key==123

I am trying to use protoc-gen-gorm to generate my services

I have setup in my proto file a message as such

message ListRequest {
    infoblox.api.FieldSelection fields = 1;
    infoblox.api.Pagination paging = 2;
    infoblox.api.Sorting order_by = 3;
    infoblox.api.Filtering filter = 4;
}

service ClientService {

    option (gorm.server).autogen = true;

   rpc List (ListRequest) returns (ListClientsResponse){
        option (google.api.http) = {
            get: "/api/clients"
        };
    };
}

I am using the patched version of gen swagger.

I also have in my server setup for my rest server

mux := runtime.NewServeMux(runtime.WithMetadata(func(ctx context.Context, req *http.Request) metadata.MD {
		return gateway.MetadataAnnotator(ctx, req)
	}))

and for the grpc server

interceptors := []grpc.UnaryServerInterceptor{
		gateway.UnaryServerInterceptor(),
	}

	log.Print("create new grpc server with config")
	server := grpc.NewServer(grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(interceptors...)))

Am I missing something?

Thank you

There isn't a way to create a new bloxid object by passing the unique id of a previously generated random scheme bloxid. In the database, we only store the unique id portion of the bloxid. We need a way to reconstitute the bloxid object. Suggestion:

add WithScheme(WithRandomEncodedID(encodedID string)) to V0 constructor option:

v0, err := NewV0("",
            WithEntityDomain("iam"),
            WithEntityType("group"),
            WithRealm("us-com-1"),
            WithSchemer(WithRandomEncodedID('tshwyq3mfkgqqcfa76a5hbr2uaayzw3h'))
        )
// v0.String(): blox0.iam.group.us-com-1.tshwyq3mfkgqqcfa76a5hbr2uaayzw3h
// v0.Encoded(): "tshwyq3mfkgqqcfa76a5hbr2uaayzw3h"
// v0.Decoded(): "9c8f6c436c2a8d0808a0ff81d3863aa0018cdb67"

Need to set up the server TLSConfig properly, including the ClientCAs and the tls.RequireAndVerifyClientCert

bloxids should have a domain field as a way to namespace domain objects, i.e.: domainX.group, domainY.group, ...

bloxids would become:

<bloxversion>.<domain>.<entity_type>.<realm>.<unique_id>

i.e.

blox0.domainX.group.us-com-1.jbeuiwrsmq3tkmzwmuzwcojsmrqwemrtgy3tqzbvhbsdizjvhe2dkn3cgzrdjyqz

i am writing a linter to lint that pass []any as any in variadic function

and run github Actions for some top go packages, this package failed

see github actions lint result here https://github.com/alingse/asasalint/runs/7284783491?check_suite_focus=true

Error: /home/runner/work/asasalint/asasalint/atlas-app-toolkit/gateway/errors.go:271:38: pass []any as any to func NewWithFields func(message string, kvpairs ...interface{}) github.com/infobloxopen/atlas-app-toolkit/gateway.MessageWithFields

func WithCodedSuccess(ctx context.Context, c codes.Code, msg string, args ...interface{}) error {
	WithSuccess(ctx, NewWithFields(msg, args))
	return SetStatus(ctx, status.New(c, msg))
}

here the args will pass as []any{args...} into NewWithFields

Atlas app toolkit uses github.com/golang/protobuf. This library is superseded by google.golang.org/protobuf.
Any repo with dependency on atlas-app-toolkit, will get an error

WARNING: Package "github.com/golang/protobuf/protoc-gen-go/generator" is deprecated. A future release of golang/protobuf will delete this package, which has long been excluded from the compatibility promise.

So, this dependency needs to be changed.

This middleware should be added early in the flow for grpc interceptors to ensure errors during authn/authz are captured.

https://github.com/grpc-ecosystem/go-grpc-prometheus

We need an easy way to run tests that will:

• generate a project
• run the server and gateway
• issue REST requests and verify the responses

Otherwise, every new commit is an unacceptable liability.

This is only support for default_order but no limit.
I think limit is also needed some times.

eg:
User -> has_many Tags(max_limit:20, default_order:"created_at")
ListUsers with _fileds='name,tags(updated_at/10)'.

Just a idea.. :)

What the title says

E.g. for the following result

{
  "result": {
    "field": "value",
    "list": [
      {
        "nested_field1": "value1",
        "nested_field2": "value1"
      },
      {
        "nested_field1": "value2",
        "nested_field2": "value2"
      }
    ]
  }
}

and field selector _fields=list.nested_field1
this should be returned:

{
  "result": {
    "list": [
      {
        "nested_field1": "value1",
      },
      {
        "nested_field1": "value2",
      }
    ]
  }
}

bloxid.V0 has all private fields and so can't be marshaled/unmarshaled into json when using it as part of a struct.
Implementing MarshalJSON and UnmarshalJSON for it should fix the issue

The term region used for bloxid makes people think of aws regions, us-east-1, us-gov-west-1, eu-central-1, ...

https://docs.google.com/spreadsheets/d/1vOV6X7J4b3b0U3A4DoS-APTHynRnezR8bJet4MsIoKQ/edit#gid=0

realm: identifiable geographic bounded area where data persistence exists
region: identifiable geographic bounded area as defined by an organizational entity - most likely defined by a cloud provider

Replace bloxid.Region with Realm.

I think doing so would make development easier because you could just go run a single package and be able to issue curl commands without having to manage two dependencies.

Thoughts?

github.com/dgrijalva/jwt-go has security issue (GHSA-w73w-5m7g-f7qc) and is recommended to replace the library with github.com/golang-jwt/jwt

https://docs.google.com/document/d/1jFk0SYh11b5y_pdiZ4OYSAUIy7Udt4NbOJ21UCxh9_g/edit#heading=h.10bf8v5i9ao3

We need to run unit, integration tests and coverage reports on PRs

Currently NGINX controller accepts X-Request-ID and auto generates an ID . Internal components who use requestID middleware looks for Request-ID. So the nginx requestID gets lost and new requestID gets propagated.

Proposals

• make changes in nginx to accept Request-ID header
  or
• Modify the middleware to accept just X-Request-ID or both.

https://github.com/golang/protobuf/releases/tag/v1.4.2

--- FAIL: TestForwardResponseMessageWithNil (0.00s)
panic: interface conversion: interface {} is nil, not map[string]interface {} [recovered]
        panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 53 [running]:
testing.tRunner.func1(0xc000307600)
        /usr/local/go/src/testing/testing.go:874 +0x3a3
panic(0x9f83c0, 0xc00030de00)
        /usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/infobloxopen/atlas-app-toolkit/gateway.TestForwardResponseMessageWithNil(0xc000307600)
        /home/drew/src/github.com/infobloxopen/atlas-app-toolkit/gateway/response_test.go:271 +0x4cd
testing.tRunner(0xc000307600, 0xae1d60)
        /usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:960 +0x350

I have included resource Identifier in auditlogging. When making a get all (list) to logs, it makes the query in the following way
SELECT * FROM "audit_logs" WHERE ("audit_logs"."account_id" = '123') AND ("audit_logs"."id" = '0') ORDER BY "id" LIMIT 1000 OFFSET 0

Because if Id isnt present in the query, we are adding "0" as value. Hence the query always returns empty.
refer: https://github.com/infobloxopen/atlas-app-toolkit/blob/master/gorm/resource/resource.go#L120

when encoded, the new BloxID format documented here cannot be converted to the original string

This leads to these filters not to work:

i.e., the following works for legacy bloxid format

_filter=id in ["hxfnyyg3qinx24g2k5qbp5origfnzpov"]
_filter=id=="hxfnyyg3qinx24g2k5qbp5origfnzpov"

but the following does not work for the new BloxID format :

_filter=id in ["blox0.infra.template.dev-0.hxfnyyg3qinx24g2k5qbp5origfnzpov"]
_filter=id=="blox0.infra.template.dev-0.hxfnyyg3qinx24g2k5qbp5origfnzpov"

I've checked the atlas contact sample app, but that seems to have been deprecated, when i try to create an starter using atlas-cli and add some custom entities, the APIs when hit through curl are throwing not implemented errors, but i see that the implementation of those methods are generated properly. Kindly help.

problem

consider the following struct

struct A {
    ID string `gorm:"type:varchar(255)"`
    BID string `gorm:"type:varchar(255)"`
    b B `gorm:"foreignkey:BID"`
}
struct B {
    ID string `gorm:"type:varchar(255)"`
    cs []*C `gorm:"foreignkey:BID"`
}
struct C {
    ID string `gorm:"type:varchar(255)"`
    BID string `gorm:"type:varchar(255)"`
}

When I using query to list A's records, preloadEverything function in gorm/fields.go will find out preloads for B and B.C; however, when gorm preloads B, it will preload C in preload function which is after query function, and preloadMap will get B and C as key, and then return. As key C != key B.C, so gorm will preload C again, and gets duplicated C's records for B.cs

confused

When I checked preloadEverything function, I am really confused for subpreload as sf.Name has also been put into toPreload function (which means subpreload will be done when gorm preload sf.Name).
So why need to preload subpreload?

Solution

I just preload sf.Name and remove subpreload and the problem of duplicated records is solved.