google / gnostic Goto Github PK

Prerequisite to Issue #7 .

URLs and filenames should be supported as input.

I believe these are all issues with the generated schema:

The definition of requestBody is an array of requestBodyOrReference - it should be a Map of Content Objects.

         "requestBody": {
-          "type": "array",
-          "items": {
-            "$ref": "#/definitions/requestBodyOrReference"
-          },
-          "uniqueItems": true
+          "type": "object",
+          "properties": {
+              "content": {
+                  "$ref": "#/definitions/content"
+              }
+          }

The definition of a content Object has additionalProperties of mediaType but should be a Map of mediaTypes.

     "content": {
       "type": "object",
       "description": "Describes a set of supported media types. A Content Object can be used in Request Body Object, Parameter Objects, Header Objects, and Response Objects.  Each key in the Content Object is the media type of the Media Type Object.",
-      "additionalProperties": {
-        "$ref": "#/definitions/mediaType"
+      "patternProperties": {
+        ".*": {
+          "$ref": "#/definitions/mediaType"
+        }
       }
     },

mediaType is missing description property.

     "mediaType": {
       "type": "object",
       "description": "Each Media Type Object provides schema and examples for a the media type identified by its key.  Media Type Objects can be used in a Content Object.",
       "patternProperties": {
         "^x-": {
           "$ref": "#/definitions/specificationExtension"
         }
       },
       "properties": {
+        "description": {
+            "type": "string"
+        },
         "schema": {
           "$ref": "#/definitions/schemaOrReference"
         },

Remove property $ref from schema, already handled

           "$ref": "http://json-schema.org/draft-04/schema#/properties/enum"
         },
-        "$ref": {
-          "type": "string"
-        },
         "type": {
           "type": "string"
         },

securityScheme remove properties from required when they only have selective validity.

     "securityScheme": {
       "type": "object",
       "description": "Allows the definition of a security scheme that can be used by the operations. Supported schemes are HTTP authentication, an API key (either as a header or as a query parameter) and OAuth2's common flows (implicit, password, application and access code).",
       "required": [
-        "type",
-        "name",
-        "in",
-        "scheme",
-        "flow",
-        "openIdConnectUrl"
+        "type"
       ],

Same for oauthFlow

     "oauthFlow": {
       "type": "object",
       "description": "Configuration details for a supported OAuth Flow",
       "required": [
-        "authorizationUrl",
-        "tokenUrl",
         "scopes"
       ],

schemaOrReference had to change this to anyOf I think because schema itself contains $ref

     "schemaOrReference": {
-      "oneOf": [
+      "anyOf": [
         {
           "$ref": "#/definitions/schema"
         },
         {
           "$ref": "#/definitions/reference"
         }
       ]

any remove type: object, should match primitives etc

     "any": {
-      "type": "object",
       "additionalProperties": true,
       "additionalItems": true
     },

specificationExtension remove type: object, allow primitive types

     "specificationExtension": {
-      "type": "object",
+      "oneOf": [
+        {
+          "type": "integer"
+        },
+        {
+          "type": "number"
+        },
+        {
+          "type": "boolean"
+        },
+        {
+          "type": "string"
+        },
+        {
+          "type": "object"
+        },
+        {
+          "type": "array"
+        }
+      ],

Please let me know if you want me to annotate them against sections of the spec. I made these changes validating the output of a 2-to-3 converter, so there may be bugs in it, or in my reading of the spec.

Thanks, Mike

While testing extensions with a real-world schema, I discovered that extensions of array and scalar types were not being handled. Here's a schema that illustrates the problem:

title: "JSON schema for apis.guru extensions"
$schema: "http://json-schema.org/draft-04/schema#"
type: object
definitions:
  Logo:
    type: object
    id: x-logo
    required:
    - url
    properties:
      url:
        type: string
        format: url
  APIClientRegistration:
    type: object
    id: x-apiClientRegistration
    required:
    - url
    properties:
      url:
        type: string
        format: url
  Origin:
    id: x-origin
    type: object
    required:
    - format
    - url
    - version
    properties:
      format:
        type: string
      url:
        type: string
        format: url
      version:
        type: string
  Preferred:
    id: x-preferred
    type: boolean
  ProviderName:
    id: x-providerName
    type: string
  ServiceName:
    id: x-serviceName
    type: string
  APISGuruCategories:
    id: x-apisguru-categories
    type: array
  Tags:
    id: x-tags
    type: array
  UnofficialSpec:
    id: x-unofficialSpec
    type: boolean

Currently the reader only reads JSON OpenAPI specifications.

The Swift code generation plugin uses features that aren't in the latest tagged version of Stencil (0.7.2). When a new version is available, test it by updating plugins/swift/openapi_swift_generator/Package.swift to use it and rebuild.

Also discussed here.

This would improve consistency and make the plugins look better if/when they branch out into standalone repos. E.g. "gnostic_go_generator" would become "gnostic-go-generator".

From the JSON schema draft 4 validation document:

Successful validation of an array instance with regards to these two
keywords is determined as follows:

if "items" is not present, or its value is an object, validation
of the instance always succeeds, regardless of the value of
"additionalItems";

Therefore is the use of additionalItems: true achieving anything? I ask because I'm trying determine how close we are to an OpenAPI-described API being able to serve its own OpenAPI definition via a schema-defined response.

I tried to run the following command to generate petstore.pb file

▶ gnostic --pb-out=. examples/v3.0/json/petstore.json

Then run report for the petstore.pb file. it gave

▶ report petstore.pb
panic: unexpected EOF

goroutine 1 [running]:
main.readDocumentFromFileWithName(0x7fff5fbff13f, 0xb, 0xc420098360)
	/Users/xxx/.gvm/pkgsets/go1.10/api/src/github.com/googleapis/gnostic/apps/report/main.go:40 +0x14d
main.main()
	/Users/xxx/.gvm/pkgsets/go1.10/api/src/github.com/googleapis/gnostic/apps/report/main.go:232 +0x98

I compiled petstore https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml

gnostic petstore.json --pb-out=.   

in my python

from sys import argv
import os
# from gnostic.discovery_pb2 import Request, Response
from gnostic.OpenAPIv3_pb2 import Document
from gnostic.plugin_pb2 import *
from gnostic.surface_pb2 import Model


if __name__ == "__main__":
    srcfile = ""
    if len(argv) > 2:
        srcfile = argv[2]
    else:
        srcfile = os.path.join(os.path.dirname(__file__), "petstore.pb")
    sourceb = open(srcfile, "rb").read()

    req = Request().FromString(sourceb)

and I get

[libprotobuf ERROR google/protobuf/wire_format_lite.cc:534] String field 'gnostic.plugin.v1.Request.output_path' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.
*** google.protobuf.message.DecodeError: Error parsing message

I also tried changing

_REQUEST = _descriptor.Descriptor(
  name='Request',
  full_name='gnostic.plugin.v1.Request',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='source_name', full_name='gnostic.plugin.v1.Request.source_name', index=0,
      number=1, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='output_path', full_name='gnostic.plugin.v1.Request.output_path', index=1,
      number=2, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),

output_path to 12 bytes instead of 9 string but the error still the same

[libprotobuf ERROR google/protobuf/wire_format_lite.cc:534] String field 'gnostic.plugin.v1.Request.output_path' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.
google.protobuf.message.DecodeError: Error parsing message but it still sees it as bytes

compiled proto

b'\n\x053.0.0\x12 \n\x10Swagger Petstore*\x05\n\x03MIT2\x051.0.0\x1a\x1f\n\x1dhttp://petstore.swagger.io/v1"\xc0\x06\n\x81\x04\n\x05/pets\x12\xf7\x03"\xe7\x02\n\x04pets\x12\rList all pets*\x08listPets2V\nT\n\x05limit\x12\x05query\x1a.How many items to return at one time (max 100)R\x14\n\x12\xca\x01\x07integer\x9a\x02\x05int32B\xed\x01\nL\nJ\n\x10unexpected error\x1a6\n4\n\x10application/json\x12 \n\x1e\x12\x1c\n\x1a#/components/schemas/Error\x12\x9c\x01\n\x03200\x12\x94\x01\n\x91\x01\n\x15A paged array of pets\x12A\n?\n\x06x-next\x125\n3\n$A link to the next page of responsesB\x0b\n\t\xca\x01\x06string\x1a5\n3\n\x10application/json\x12\x1f\n\x1d\x12\x1b\n\x19#/components/schemas/Pets2\x8a\x01\n\x04pets\x12\x0cCreate a pet*\ncreatePetsBh\nL\nJ\n\x10unexpected error\x1a6\n4\n\x10application/json\x12 \n\x1e\x12\x1c\n\x1a#/components/schemas/Error\x12\x18\n\x03201\x12\x11\n\x0f\n\rNull response\n\xb9\x02\n\r/pets/{petId}\x12\xa7\x02"\xa4\x02\n\x04pets\x12\x17Info for a specific pet*\x0bshowPetById2=\n;\n\x05petId\x12\x04path\x1a\x1dThe id of the pet to retrieve \x01R\x0b\n\t\xca\x01\x06stringB\xb6\x01\nL\nJ\n\x10unexpected error\x1a6\n4\n\x10application/json\x12 \n\x1e\x12\x1c\n\x1a#/components/schemas/Error\x12f\n\x03200\x12_\n]\n$Expected response to a valid request\x1a5\n3\n\x10application/json\x12\x1f\n\x1d\x12\x1b\n\x19#/components/schemas/Pets*\xee\x01\n\xeb\x01\n]\n\x03Pet\x12V\nT\xba\x01\x02id\xba\x01\x04name\xfa\x01E\n\x1a\n\x02id\x12\x14\n\x12\xca\x01\x07integer\x9a\x02\x05int64\n\x13\n\x04name\x12\x0b\n\t\xca\x01\x06string\n\x12\n\x03tag\x12\x0b\n\t\xca\x01\x06string\n3\n\x04Pets\x12+\n)\xca\x01\x05array\xf2\x01\x1e\n\x1c\x12\x1a\n\x18#/components/schemas/Pet\nU\n\x05Error\x12L\nJ\xba\x01\x04code\xba\x01\x07message\xfa\x016\n\x1c\n\x04code\x12\x14\n\x12\xca\x01\x07integer\x9a\x02\x05int32\n\x16\n\x07message\x12\x0b\n\t\xca\x01\x06string'

This is to follow the common convention of naming compilers by appending the language name with "c" (protoc, swiftc, etc).

There's currently no way to go back from the proto representation to JSON or YAML.

We should add one.

Hi there! Since OAS 3.0.1 is now published, are there any plans to rev gnostic's current draft support of the spec?

The OpenAPI schema indicates that some of the properties of many object types are required. Enhance the compiler to verify that these required properties are present and if not, provide specific error messages indicating what properties should be added.

Test files live inside directory:

https://github.com/googleapis/api-compiler/tree/master/src/test/java/com/google/api/tools/framework/importers/swagger/testdata

library_example_yaml.yaml
additional_properties.json
top_level_security_ext.json
x_google_allow_extension_all.json
x_google_allow_extension_all_with_existing_catchall_methods.json

@timburks May be the change you are making currently to make some real world swagger files work, would automatically fix this problem.

It would be good for all gnostic-related executables to be consistently named (i.e. share the "gnostic-" prefix) and this would clarify the purpose of "generator", which can now be used by third parties to generate extension handlers from the JSON Schemas of their extensions.

Expected behavior

It should be possible to run go get against the gnostic repo using ... to follow all dependencies as well, e.g.

go get github.com/googleapis/gnostic/plugins/...

Actual behavior

Running go get github.com/googleapis/gnostic/plugins/... fails with this error:

$ go get github.com/googleapis/gnostic/plugins/... 
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/apis_guru/apis_guru: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/apis_guru/apis_guru" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/apis_guru/apis_guru (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/apis_guru/apis_guru (from $GOPATH)
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/bookstore/bookstore: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/bookstore/bookstore" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/bookstore/bookstore (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/bookstore/bookstore (from $GOPATH)
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/sample/sample: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/sample/sample" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/sample/sample (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/sample/sample (from $GOPATH)
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/xkcd/xkcd: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/xkcd/xkcd" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/xkcd/xkcd (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v2.0/xkcd/xkcd (from $GOPATH)
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/bookstore/bookstore: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/bookstore/bookstore" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/bookstore/bookstore (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/bookstore/bookstore (from $GOPATH)
package github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/urlshortener/urlshortener: cannot find package "github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/urlshortener/urlshortener" in any of:
	/usr/lib/google-golang/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/urlshortener/urlshortener (from $GOROOT)
	/usr/local/google/home/christiewilson/Code/go/src/github.com/googleapis/gnostic/plugins/gnostic-go-generator/examples/v3.0/urlshortener/urlshortener (from $GOPATH)

This is because the go generator plugin examples rely on generated code which is not present in the repo.

Steps to reproduce

Run go get github.com/googleapis/gnostic/plugins/....

Currently extensions are represented with an internal "Any" type that stores them in the model as JSON. I've heard other ideas for this, including representing them as parsed protobufs defined by other schema.

OpenAPI 3 supports empty schema (see https://swagger.io/docs/specification/data-models/data-types/#any)

Empty schema allows for flexibility - be strict in some parts of the schema/API while still allowing some properties to be of "any" type. E.g. for APIs which act as proxy, dumb pipe like PubSub kind of service, it's possible to produce flexible API, allowing message/event source and end consumer to agree on the data (message payload) schema (and just pass that info through), while same API could still have strict enforcing of some mandatory context/metadata information.

Currently gnostic-go-generator for an empty schema generates an empty struct type. IMO it should generate interface type instead.

By https://golang.org/pkg/encoding/json/#Marshal

Struct values encode as JSON objects.

while

Interface values encode as the value contained in the interface.

Similarly, by https://golang.org/pkg/encoding/json/#Unmarshal

To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal ...

while

To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:

bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

If gnostic-go-generator would generate interface type instead of struct type for empty schema then e.g. in PubSub kind of API implementation JSON property of "any" type could be properly passed through (unmarshalled and marshalled) without losing data.

Current workaround is to manually change the type for AnyValue in generated code (types.go).

The compiler should support a plugin interface similar to protoc. Ideally it would be durable to future version changes. One way to provide this would be to describe the API being compiled in a field of type “Any” inside a “PluginRequest” message.

This should also include a simple sample plugin.

Paging @tfesenko and @timburks

The regular expression for scope names has been set to ^[a-zA-Z0-9\\.\\-_]+$ which matches what {name} is set to for elements under the components property.

I believe this is too restrictive in this case. As far as I can see scope names are only limited by their definition in RFC6749.

I'm not sure how JSON schema patternProperties disambiguation is supposed to work when more than one regex matches (in this instance ^x- would also match a valid scope name.

The following piece of code is located at plugins/environments.go:

		if err == nil {
			env.Request.AddModel("openapi.v2.Document", documentv2)
			// include experimental API surface model
			surfaceModel, err := surface.NewModelFromOpenAPI2(documentv2)
			if err != nil {
				env.Request.AddModel("surface.v1.Model", surfaceModel)
			}
			return env, err
		}
		// If that failed, ignore deserialization errors and try to unmarshal OpenAPI v3.
		documentv3 := &openapiv3.Document{}
		err = proto.Unmarshal(apiData, documentv3)
		if err == nil {
			env.Request.AddModel("openapi.v3.Document", documentv3)
			// include experimental API surface model
			surfaceModel, err := surface.NewModelFromOpenAPI3(documentv3)
			if err != nil {
				env.Request.AddModel("surface.v1.Model", surfaceModel)
			}
			return env, err
		}

Apparently the inner if judgement is a bug which should be like this:

	if err == nil {
		env.Request.AddModel("surface.v1.Model", surfaceModel)
	}

Please fix it to be able to genenrate code for openapi.v2.Document and openapi.v3.Document.

Since the OpenAPI spec is read as JSON using the standard Go JSON reader and represented with Go maps, the ordering of values in the source files is lost.

Currently errors are reported in terms of key paths in the source documents. It would be helpful to users to have this reflected back to line numbers.

With this issue, we need an official Travis job that build the code and runs that tests on every pull request.

Hi, I'm super excited to use this pkg, but I'm having lots of issues. I'd also like to help where I can.

I'm using this open api v3 spec https://us-east-1-production-zcv2-api.ziff.ai/_schema

first problem is:

ERROR $root.components.schemas.cloud.schema.FileEncoder.schema.properties.encoderModelUri.schema has invalid property: $ref
ERROR $root.components.schemas.cloud.schema.FileEncoder.schema.properties.encoderModelUri.reference has invalid property: readOnly
ERROR $root.components.schemas.cloud.schema.FileEncoder.reference is missing required property: $ref
ERROR $root.components.schemas.cloud.schema.FileEncoder.reference has invalid properties: type, required, properties

As I wasn't using that model, I just removed that one field in the json spec.

My next issue was

2019/02/07 09:22:57 unimplemented: any_of:<schema:<type:"string" > > any_of:<schema:<type:"integer" format:"int64" > > any_of:<schema:<type:"number" format:"double" > >
2019/02/07 09:22:57 unimplemented: any_of:<schema:<type:"integer" format:"int64" > > any_of:<schema:<type:"string" > >

I understand that anyOf types would be hard to implement in go, I just changed these to type string

gnostic ziff-api.json --go-generator-out=ziff then ran without errors but types.go and client.go were completely empty.

Currently they hang until their input is closed and then crash when they fail to deserialize a protobuf.

I have an OpenAPIv3 schema or JsonSchema.
Can I use gnostic or gnostic-generate to produce equivalent .proto file?

openapi: 3.0.0
info:
  title: Sample API
  version: 0.1.9
paths:
  /pipeline_runs:
    post:
      requestBody:
        content:
          application/json:
            schema:
              $ref: https://raw.githubusercontent.com/kubeflow/pipelines/618b559391a225a6d821141047d6f7438e6f64a1/sdk/python/kfp/components/structures/components.json_schema.json#/definitions/TaskSpec
      responses:
        '200':
          description: Updated

These links are currently installed into $(GOPATH):
ln -s $(GOPATH)/bin/openapi_go_generator $(GOPATH)/bin/openapi_go_client

But GOPATH can be a ":"-separated list of paths, and that breaks this process.

Instead modify the installation to link in the same directory that contains openapi_go_generator, which can be obtained as follows:

GENERATOR = `which openapi_go_generator`
GOBIN = `basepath $GENERATOR`
ln -s $(GOBIN)/openapi_go_generator $(GOBIN)/openapi_go_client

See the fields named "description" in the OpenAPI schema.

These should be included as comments alongside the corresponding generated field descriptions.

First thanks for working on a Golang project that understands/parses/validates multiple API specs!

I am using gnostic to parse & validate OpenAPIv3 specs (and later to validate API responses).
When translating from Gnostic's generated openapiv3.proto to my own intermediary format I noticed that some values have dubious defaults, such as OpenAPI's schema's minimum field:

https://github.com/googleapis/gnostic/blob/master/OpenAPIv3/OpenAPIv3.proto#L549

https://github.com/googleapis/gnostic/blob/master/OpenAPIv3/OpenAPIv3.go#L8100-L8102

IMO a boolean has_minimum should be introduced here, as a minimum of 0 is meaningful for real numbers.

Another note: it seems that OAS-flavored JSON schemas are not validated. They can contain type: 'blipblop' without issue.
Did I miss something? Is this planned to be supported?

I'm happy to open PRs!

I would also be very happy to see other Golang libs you like that

• parse at least OpenAPI specs
• validate the full spec (including OAS-flavored schemas), displaying useful warnings/errors
• can validate a net/http response against a spec

I'm building https://fuzzymonkey.co's client in Go but really the most complete libs seem to be in JS...

Quote from OpenAPI 3 Schema Object specification

items - Value MUST be an object and not an array. Inline or referenced schema MUST be of a Schema Object and not a standard JSON Schema. items MUST be present if the type is array.

But now, in gnostic v3 model Items are array, what is make usage less obvious.
Seems gnostic use only first item only anyway.

We are using this repo's proto definitions of OpenAPI v2 in kubernetes. In MIME type, we reference this repo and it would be nice if we have a release branch to point instead of a hash.

https://github.com/kubernetes/kube-openapi/blob/c2fb93ef0d03d716a5a7d0df759d693b9ff2e07c/pkg/handler/handler.go#L48

An alternative would be moving proto files to a separate repo, something like proto-openapi in rhyme with go-openapi?

I know the docs say that -out is also the package name, but the error message is confusing. when I do something like -output=/tmp/bookstore I get

Plugin error: [constants.go:3:9: expected 'IDENT', found '/']

I think it would be nice to have a friendlier error.

I've traced this to goimports errors. related to #110
https://github.com/googleapis/gnostic/blob/d55a06a32dc7468c645ac01d93d37d4dd47ca062/plugins/gnostic-go-generator/renderer.go#L63

I made some changes like

diff --git a/plugins/gnostic-go-generator/renderer.go b/plugins/gnostic-go-generator/renderer.go
index 93fd47d..31efb8e 100644
--- a/plugins/gnostic-go-generator/renderer.go
+++ b/plugins/gnostic-go-generator/renderer.go
@@ -60,7 +60,9 @@ func (renderer *Renderer) Render(response *plugins.Response, files []string) (er
                }
                // run generated Go files through imports pkg
                if filepath.Ext(file.Name) == ".go" {
-                       file.Data, err = imports.Process(file.Name, file.Data, nil)
+                       if file.Data, err = imports.Process(file.Name, file.Data, nil); err != nil {
+                               response.Errors = append(response.Errors, fmt.Sprintf("ERROR %v", err))
+                       }
                }
                response.Files = append(response.Files, file)
        }

The last error gets reported twice though due to named returns, and I was thinking it would be really nice to still write out the files so that the user can inspect the problem in context.

thoughts?

go test fails in tools/j2y2j on release 0.1.0

with undefined: jsonschema.Render

OpenAPI descriptions with large numbers of repeated $refs can take a long time to resolve. This is because we explicitly expand all $refs.

For example, this runs much faster with --keep_refs than without it:
openapic --sample_out=- https://api.apis.guru/v2/specs/bitbucket.org/2.0/swagger.yaml --keep_refs

A better approach would be to keep $refs in place but still expand each unique $ref and store the expansion in the compiled protos so it could be easily used in client code when needed.

Create an example that shows code generation using openapic.

when using gnostic to parse the openAPIv3 document, our code is as follow:

func unmarshalApiSpec(svcName string, apiSpecStr string) (*openapi_v3.Document, error) {
	decodeValue, err := Base64Decode(apiSpecStr)
	if nil != err {
		return nil, err
	}
	rawText, err := DeCompressDefinition(decodeValue)
	if nil != err {
		return nil, err
	}
	info, err := compiler.ReadInfoFromBytes(svcName, rawText)
	if err != nil {
		return nil, err
	}
	doc, err := openapi_v3.NewDocument(info, compiler.NewContext("$root", nil))
	if err != nil {
		return nil, err
	}
	return doc, nil
}

We encounter the problem that: when we call unmarshalApiSpec twice using the same svcName, but different apiSpecStr, it is always return the first parse result unless you restart the process.
As we study the code, we found that gnostic has cache for cache the unmarshal result, as follow:

func ReadInfoFromBytes(filename string, bytes []byte) (interface{}, error) {
	initializeInfoCache()
	cachedInfo, ok := **infoCache[filename]
	if ok {
		if verboseReader {
			log.Printf("Cache hit info for file %s", filename)
		}
		return cachedInfo, nil
	}
	if verboseReader {
		log.Printf("Reading info for file %s", filename)
	}
	var info yaml.MapSlice
	err := yaml.Unmarshal(bytes, &info)
	if err != nil {
		return nil, err
	}
	if len(filename) > 0 {
		infoCache[filename] = info
	}
	return info, nil
}

We did not find any methods to clear or disable this cache.

I tried the petstore example. Pets which is supposed to be an array of Pet, is being generated as an empty struct. Here is a dump of the Type

   (*surface_v1.Type)(0xc00006e960)({
    Name: (string) (len=4) "Pets",
    Kind: (surface_v1.TypeKind) 0,
    Description: (string) (len=41) "implements the service definition of Pets",
    ContentType: (string) "",
    Fields: ([]*surface_v1.Field) <nil>,
    TypeName: (string) (len=4) "Pets"
   }),

When a linter runs and finds no problems, if it doesn't return any messages, a user might think that it didn't run at all.

Can we fix this by having each linter write a summary message?
Would it be better to handle this in gnostic?

Determine appropriate values for the following code generation options and add them (via @wora).

// This option lets the proto compiler generate Java code inside the package
// name (see below) instead of inside an outer class. It creates a simpler
// developer experience by reducing one-level of name nesting and be
// consistent with most programming languages that don't support outer classes.
option java_multiple_files = true;

// The Java outer classname should be the filename in UpperCamelCase. This
// class is only used to hold proto descriptor, so developers don't need to
// work with it directly.
option java_outer_classname = "XyzProto";

// The Java package name must be proto package name with proper prefix.
option java_package = "com.company.abc.xyz.v1";

// A reasonable prefix for the Objective-C symbols generated from the package.
// It should at a minimum be 3 characters long, all uppercase, and convention
// is to use an abbreviation of the package name. Something short, but
// hopefully unique enough to not conflict with things that may come along in
// the future. 'GPB' is reserved for the protocol buffer implementation itself.
//
option objc_class_prefix = "GABCX";

Empty object responses are causing gnostic to produce invalid go stanzas:

Given a schema response such as:

       responses:
         '200':
           description: Any endpoint that yields "{}"
           content:
             application/json:
                schema:
                 type: object

gnostic produces a client method whose result is:

result := &interface{}{}

I've also tried adding the implicit: properties: {}, but the generated client includes the same bug (expected ';', found '{').

More details can be found here, https://gist.github.com/displague/31700ba03009d1a183297949c54e7471 including the schema being used and the full client output.

We propose using "openapi.v2" instead of OpenAPIv2 (via @wora).

// The package name should start with the company name and end with
// the major version.
package company.abc.xyz.v1;

Currently the output files are dropped inside the current working directory of the process. Ability to pass the output directory or output file path will be handy especially when calling this tool from another application through process.start (out of proc).

As by https://swagger.io/docs/specification/describing-responses/

An API specification needs to specify the responses for all API operations. Each operation must have at least one response defined, usually a successful response. A response is defined by its HTTP status code and the data returned in the response body and/or headers.

https://github.com/googleapis/gnostic/blob/master/plugins/gnostic-go-generator/language.go#L89 handles well only "200" response by using OK as field name. For all other responses, response code is being used as field name and since response codes are numbers they do not satisfy Go's requirement that variable names have to start with a letter. So types.go and client.go get generated with syntax errors.

Valid OpenAPI documents may become invalid after decode and encode because empty required fields are omitted.
Seems should be fixed in ToRawInfo(), by getting rid of if <not empty> { } conditions, for fields that are required by OpenAPI specification. Personally, I have problem with description of ResponseObject.
Failing test, that expected not to fail:

package empty_test

import (
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"gopkg.in/yaml.v2"

	"github.com/googleapis/gnostic/OpenAPIv2"
	"github.com/googleapis/gnostic/compiler"
)

const docEmpty = `swagger: "2.0"
info:
  version: v9
  title: ""
  description: ""
paths:
`

func TestEmpty(t *testing.T) {
	t.Log("Original decode")
	doc, err := decodeDoc(t, docEmpty)
	require.NoError(t, err)
	encoded := encodeDoc(t, doc)
	t.Log("Decode decoded and encoded")
	_, err = decodeDoc(t, encoded)
	assert.NoError(t, err, "decode encoded failed :(")
	assert.Equal(t, docEmpty, encoded)
}

func encodeDoc(t *testing.T, doc *openapi_v2.Document) string {
	bytes, err := yaml.Marshal(doc.ToRawInfo())
	require.NoError(t, err)
	return string(bytes)
}

func decodeDoc(t *testing.T, data string) (*openapi_v2.Document, error) {
	var info yaml.MapSlice
	err := yaml.Unmarshal([]byte(data), &info)
	require.NoError(t, err)
	t.Log("After yaml parse: ", info)
	return openapi_v2.NewDocument(info, compiler.NewContextWithExtensions("$root", nil, nil))
}

Output:

	empty_test.go:23: Original decode
	empty_test.go:43: After yaml parse:  [{swagger 2.0} {info [{version v9} {title } {description }]} {paths <nil>}]
	empty_test.go:27: Decode decoded and encoded
	empty_test.go:43: After yaml parse:  [{swagger 2.0} {info [{version v9}]}]
	Error Trace:	empty_test.go:29
	Error:      	Received unexpected error:
	            	ERROR $root is missing required property: paths
	            	ERROR $root.info is missing required property: title
	Test:       	TestEmpty
	Messages:   	decode encoded failed :(
	Error Trace:	empty_test.go:30
	Error:      	Not equal: 
	            	expected: "swagger: \"2.0\"\ninfo:\n  version: v9\n  title: \"\"\n  description: \"\"\npaths:\n"
	            	actual  : "swagger: \"2.0\"\ninfo:\n  version: v9\n"
	Test:       	TestEmpty

OpenAPI specifications can be divided into multiple files. We need to extend the reader to support this.

Hi all,
Thanks for the tool.
When generating a type from a schema which states a boolean property, the generator for golang produces a "Boolean" instead of "bool" native type.

components:    
  schemas:
    agent:
      type: object
      required:
        - name
        - active
      properties:
        name:
          type: string
          minLength: 4
          readOnly: true
        active:
          type: boolean

With this spec, the generator creates types.go which contains the following:

// implements the service definition of agent
type Agent struct {
	Name   string  `json:"name,omitempty"`
	Active Boolean `json:"active,omitempty"`
}

So "Boolean" is not the native golang type.

The README for step 8 is deprecated. Trying to run:
go install github.com/googleapis/gnostic/plugins/gnostic-go-sample

This should be changed to:
go install github.com/googleapis/gnostic/plugins/gnostic-go-generator

I'd be happy to fix that.

Question about object properties from @wora:

Does OpenAPI depend on presence?
e.g. for a property like "bool foo", does OpenAPI treat true, false, and null as 3 separate states?
If you use proto3, you will prohibit swagger from allowing such tri-state properties. To support tri-state properties, you have to use proto2.