Code Monkey home page Code Monkey logo

apikana's Introduction

A swisspost project

Apikana

Build project

Integrated tools for REST and Messaging API design.

Apikana combines the following tools to facilitate the authoring of contract-first REST APIs:

It basically generates formal schemas and documentation from a mixed swagger/typescript definition that is easy to author and maintain.

It supports also java:

Serialization/Deserialization of java objects:

  • The implementation needs a jackson module for serializing and deserializing the objects as described here.

Usage

Create a new API project

Install apikana npm install -g apikana. Run apikana init.

This starts an interactive wizard that lets you define the main aspects of the API project.

Then enter the project directory and run npm install to install all the required dependencies.

Use as a global tool

When apikana start is executed, it looks in src/openapi for a file named api.yaml. This is an OpenAPI 2.0 file defining the REST API. In the definitions section a $ref can be given which references typescript file(s) defining the data models. $ref can be a comma or newline separated string or an array thereof. The models should be defined as typescript export interfaces.

At the end, the dist directory contains the json schemas and a complete HTML documentation of the API. Just open a browser at http://localhost:8333.

src/openapi/api.yaml

paths:
  /sample/users:
    get:
      operationId: getUser
      responses:
        200:
          description: ok
          schema:
            $ref: "#/definitions/User"
definitions:
  $ref: ../ts/user.ts

src/ts/user.ts

export interface User {
    id: number
    firstName: string // The given name
    lastName: string // the family name @pattern [A-Z][a-z]*
    age?: number
}

Annotations like @pattern can be used to specify more precise constraints. They correspond to the JSON Schema validation keywords.

The src/style directory can contain css and image files which can be used to style the generated HTML document.

The gen directory contain the generated files relative to the enabled plugins. This files can be overwritten by defining a templates directory in the root folder of the project using the following directory structure: root_directory/templates/plugin_name/gen/plugin_name/filename.ext where:

  • root_directory is the root directory of the project,
  • plugin_name is the plugin name (for example maven or dotnet) and
  • filename.ext is the file to copy in the gen directory (for example pom.xml or api.csproj).

Matching filenames will be overwritten. All others will be copied in the gen directory.

A note on Java code generation

By default Apikana generates Maven SNAPSHOT version for release candidates (version number of the stream api looks like 1.0.0-rc.3). Due to a bug in Apikana < 0.9.23 it also considered versions like 1.0.0-feature-test.13 as release candidates. This old buggy behavior can be restored by configuring a setting in the generated stream api package.json file:

// File package.json
{
  // ...
  "customConfig": {
    // ...
    "snapshotVersion": "ALL_NON_FINAL"
    // ...
  }
}

Use as a devDependency

Instead of being globally installed, apikana can also be defined as a devDependency of a project. A sample configuration would look like:

{
  "name": "My API project",
  "scripts": {
    "start": "apikana start src"
  },
  "devDependencies": {
    "apikana": "0.7.1"
  }
}

Then simply run npm run start.

Development

Development is done within feature branches in forked repositories. When ready it gets merged to swisspost/develop via merge request (at best including review).

Make sure to comply with the conventional commits specification when writing commit or squash commit messages, as they control the next version when releasing!

Testing

You can run tests using npm test within projects root directory.

Releasing

Releasing is done automatically using semantic-release when merging to develop and master.

Merging to develop will release to the next distribution channel on npm, merging to master will publish that release to the main distribution channel.

IMPORTANT

If the conventional commits result in a new release (i.e. having feat:, fix: or BREAKING in a message), merging to master will trigger a new release on npmjs.org automatically without any further user interaction!

Publishing

To publish to npmjs.org locally, the environment variable NPM_TOKEN must be set. You can accomplish this by executing npm login locally and afterwards extracting corresponding value from ~/.nmprc.

apikana's People

Contributors

cococonscious avatar geekrumper avatar hiddenalpha avatar lbovet avatar leonardostaffolani avatar mattiamato avatar meggima avatar naamor avatar nidi3 avatar schaermu avatar stefan-schaermeli avatar swisspost-bot avatar thomastopuz avatar thwint avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

apikana's Issues

Unify trailing slashes in generated java constants

It seems we've no convention when generated java constants have trailing slashes and when they don't. Even old and new Path classes are inconsistent. That introduced lot extra work for our services upgrading the APIs and still gets us in trouble where those inconsistencies result in unexpected paths and most time aren't discovered until malformed services reach the productive stage.

My suggestion:

  • ALWAYS use trailing slashes for collections
  • NEVER use trailing slashes for resources
  • NEVER use leading slashes

Originally we expected old and new Paths are fully compatible. But in table below we collected some examples and now see some (for us evil) differences. Our expectation was that each pair for leading and trailing will be same for old and new Paths and there's also some consistency within same version. But our expectation seems wrong.

Path Type leading old leading new trailing old trailing new
mediamessage/v1/config/ Collection no no yes no
mediamessage/v1/config/suppressmessageflow Resource no no no no
suppressmessageflow Resource no yes no no
mediamessage/v1/online/{mediaChannel}/ Collection no no yes no
mediamessage/v1/online/{mediaChannel}/{stopId}/ Collection no no yes no
mediamessage/v1/online/{mediaChannel}/{stopId}/{stopIndex} Resource no no no no
{stopIndex} Resource no yes no no
stopIndex Resource no no no no
{stopId} Collection no yes no no
stopId Collection no no no no
{stopId}/{stopIndex} Resource no yes no no
{mediaChannel}/{stopId}/{stopIndex} Resource no yes no no

Apikana Maven plugin fails with Apikana > 0.7.4

When using the Apikana maven plugin the build fails, when using an Apikana 0.7.5 or greater.

It seems that generating of json-schema-3/event.json is broken. This only happens when an API uses one or more other dependent API.

Version 0.7.5 generates the following lines in event.ts:

    "driverSignon": {
      "$ref": "../../../api-dependencies/ts//vehicleoperation-recording-api/target/api-dependencies/json-schema-v3/driver-signon.json",
      "description": "the current signed in Driver on the vehicle"
    },

The part ts//vehicleoperation-recording-api/target/api-dependencies/ seems to be wrong as it does not point to the correct relative location on the filesystem.

Apikana 0.7.4 generates these lines correctly

    "driverSignon": {
      "$ref": "../../../api-dependencies/json-schema-v3/user-registration-api/driver-signon.json",
      "description": "the current signed in Driver on the vehicle"
    },

The following line in Apikana is responsible for the change:
v0.7.4...v0.7.5#diff-8793a8289addf2b230a55060bd229d406b829d97cae2c8593e32a4b9fa6070a9R61

2nd generation paths wear leading slashes where 1st generation don't

The 2nd generation Paths (as in: com.example.mypackage.MyPackagePaths) now include a leading slash where 1st generation paths (as in: com.example.mypackage.Paths) don't.

In PAISA (Project where apikana was born??) in Alice Library is defined:

These URLs also include the context path (always with trailing slash), i.e. the service name as first path element. E.g: http://example.com:1234/eagle/, http://example.com:1234/saturn/

Now as soon services migrating to 2nd gen paths they'll get in trouble because gateleen cannot handle such double slashes in its hooking module.

Workaround: Use deprecated 1st generation Paths.

Originating issue is ISAGD-5352 (private issue tracker).

References to definitions not resolved when using complete-api.yaml/json

When I use the generated complete-api.yaml or complete-api.json in Swagger 2.X, references in subobjects are not resolved and show up as text like #/definitions/ReferencedTypeName.

See the following example.

When replacing #/definitions/ReferencedTypeName with ReferencedTypeName it works.

The generated schema works against latest Swagger version, but shows some warnings and errors in Swagger Editor

I noticed that the complete-api.yaml generated using older version 0.9.21 looks entirely different and subreferences are resolved as desired. The generation with version 0.9.22 and later versions has that flaw described above.

Failure to inject script tags in index.html

Since some new version of dependencies (caused by wrongly managed package-lock.json), gulp-inject fails to inject when start/end tags are comments.
The symptom is the absence of the global spec variable:

Uncaught ReferenceError: spec is not defined
    at HTMLDocument.<anonymous> (index.html?url=/src/openapi/api.yaml:57)
    at f (jquery-1.8.0.min.js:1)
    at Object.fireWith [as resolveWith] (jquery-1.8.0.min.js:1)
    at Function.ready (jquery-1.8.0.min.js:1)
    at HTMLDocument.pe (jquery-1.8.0.min.js:1)
(anonymous) @ index.html?url=/src/openapi/api.yaml:57
f @ jquery-1.8.0.min.js:1
fireWith @ jquery-1.8.0.min.js:1
ready @ jquery-1.8.0.min.js:1
pe @ jquery-1.8.0.min.js:1

One one hand, the package-lock.json should be fixed to have no open version ranges.
On the other hand, the injection can be fixed using non-comments start/end tags.

Incompatible with recent Node.js version (12.16.0)

Throws error after trying to run apikana 0.7.6 with Node 12.16.0 (includes npm 6.13.4) on Windows 10 x64

Reproduce:

  1. Install Node.js LTS 12.16.0 (includes npm 6.13.4)
  2. Install apikana 0.7.6 with npm
  3. Create new project

apikana init

  1. Try to run

npm start

Workaround:
Downgrade Node.js i.e. Node.js 10.15.1 / npm 6.4.1

Stacktrace:

morit@DESKTOP-LLH1ME3 MINGW64 ~/dev/apikTest/spielplatz-stream-api
$ npm start

> @moritz/[email protected] start C:\Users\morit\dev\apikTest\spielplatz-stream-api
> apikana start src dist

[10:04:07]                                        AAPIKA
[10:04:07]      PIKANAAPIKANA    ANA            NAA    NA
[10:04:07]                NAA    NAA            AAP    AA
[10:04:07]          AP    AAP    AAP    AAPI      IKANAA
[10:04:07]          PI   AAP     APIKANAAP
[10:04:07]          IK  AAP      PIKAN
[10:04:07]          KANAAP       IKA
[10:04:07]         KAN           KAN
[10:04:07]         ANA           ANA
[10:04:07]        ANA            NAA
[10:04:07]      KANA              APIKANAAPI
[10:04:07]     KAN                 IKANAAPIKA        apikana 0.7.6
[10:04:07]
[10:04:07] Source:  src
Loading defaults... found 0.0.0
(node:26816) UnhandledPromiseRejectionWarning: ReferenceError: primordials is not defined
    at fs.js:35:5
    at req_ (C:\Users\morit\dev\apikTest\spielplatz-stream-api\node_modules\apikana\node_modules\natives\index.js:143:24)
    at Object.req [as require] (C:\Users\morit\dev\apikTest\spielplatz-stream-api\node_modules\apikana\node_modules\natives\index.js:55:10)
    at Object.<anonymous> (C:\Users\morit\dev\apikTest\spielplatz-stream-api\node_modules\apikana\node_modules\vinyl-fs\node_modules\graceful-fs\fs.js:1:37)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Module.require (internal/modules/cjs/loader.js:1043:19)
    at require (internal/modules/cjs/helpers.js:77:18)
(node:26816) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:26816) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Seem To Generate Somewhat Ugly Java Classes For Maps

I shortly tried to specify a type as a Map<string, Foo>. But I found no way to get proper Java types created.

As described by ekimmai on reddit.com I tried:

export interface Foo {}

export interface Bar {
    somethingElse : string;
    myMap : {[key: string]: Foo};
}

But using this I got somewhat strange java classes. I'm not sure if this is expected behavior (dropped some bloat):

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonPropertyOrder({
    "somethingElse",
    "myMap"
})
public class Bar implements Serializable
{

    @NotNull
    @JsonProperty("somethingElse")
    private String somethingElse;

    @NotNull
    @JsonProperty("myMap")
    @Valid
    private MyMap myMap;

    @JsonProperty("somethingElse")
    public String getSomethingElse() {
        return somethingElse;
    }

    @JsonProperty("somethingElse")
    public void setSomethingElse(String somethingElse) {
        this.somethingElse = somethingElse;
    }

    public Bar withSomethingElse(String somethingElse) {
        this.somethingElse = somethingElse;
        return this;
    }

    @JsonProperty("myMap")
    public MyMap getMyMap() {
        return myMap;
    }

    @JsonProperty("myMap")
    public void setMyMap(MyMap myMap) {
        this.myMap = myMap;
    }
}

And:

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonPropertyOrder({

})
public class MyMap {

    @JsonIgnore
    @Valid
    private Map<String, Foo> additionalProperties = new HashMap<String, Foo>();

    @JsonAnyGetter
    public Map<String, Foo> getAdditionalProperties() {
        return this.additionalProperties;
    }

    @JsonAnySetter
    public void setAdditionalProperty(String name, Foo value) {
        this.additionalProperties.put(name, value);
    }

}

Now I get an additional intermediate type MyMap (which in reality doesn't exist) where I need to access values through (IMHO far too long) method chain getAdditionalProperties().get("myKey").

Why don't we simply generate a class like below in this case? Eg:

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonPropertyOrder({
    "somethingElse",
    "myMap"
})
public class Bar extends LinkedHashMap<String, Foo>
{

    @NotNull
    @JsonProperty("somethingElse")
    private String somethingElse;

    @JsonProperty("somethingElse")
    public String getSomethingElse() {
        return somethingElse;
    }

    @JsonProperty("somethingElse")
    public void setSomethingElse(String somethingElse) {
        this.somethingElse = somethingElse;
    }
}

This would make that additional MyMap type superfluous and would make access far more intuitive as in .get("myKey").

Apikana > 0.9.5 fails to find api.yaml when usint maven plugin

With version 0.9.6 a new line referencing api.yaml was introduced (https://github.com/swisspush/apikana/blob/develop/bin/apikana#L118). This line uses currentPath and hard coded path to ./src/openapi/api.yaml.
When using the Apikana Maven plugin (apikana-java) apikana is started using source and api parameter

apikana start ../.. --api=src/rest/openapi/api.yaml --models=src/model/ts --target=../api --style=src/style --javaPackage=ch.post.it.paisa.api.vehiclelink.power.v1 --deploy=false --port=8333 --serve=true --openBrowser=false --config=properties.json --dependencyPath=../api-dependencies --minVersion=0.9.17 --log=info --pathPrefix=vehiclelink/power/v1/ --basePath=/ --generate3rdGenPaths=true

In this case apikana fails with an error mentioning that it is not able to find the api.yaml file:

[09:57:36] Source:  ../..
fs.js:114
    throw err;
    ^

Error: ENOENT: no such file or directory, open '/c/Work/projects/ISA/vehiclelink-power-api/target/node/src/openapi/api.yaml'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at generate (/usr/lib/node_modules/apikana/bin/apikana:122:33)
    at Object.<anonymous> (/usr/lib/node_modules/apikana/bin/apikana:32:9)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)

In the given Java project the api.yaml file is located relative to the current path:
../../src/rest/openapi/api.yaml

Chrome does not like XHR in main thread

We can see this warning in the console:

browserify.js:131919 [Deprecation] Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.

In fact, in associations with CORS headers, this activates a bug in Chrome preventing certain URLs to be fetched using sync XHR like the .ts files.
The solution is to move the generation code into a web worker.

Maven package version number is truncated

I want to implement publishing beta packages for our apikana generated NuGets, Mavens and Npms when building feature branches.

For NuGet and NPM we use the schema {semantic_version_number}-{branch-name}-{buildNr}

Example: 3.2.1-feature-new-field-2

When I tried to to this for Maven packages I noticed that part of the version name is cut and replaced.
The above example yields 3.2.1-feature-SNAPSHOT. This can result in non-unique maven packages and result in package conflicts.

The problem is the following line: https://github.com/swisspush/apikana/blob/592f18b98eb68a10078ea478ea40607aedcd316e/src/plopfile_start.js#L12

Two questions:

  • what was the original idea behind the code referenced above?
  • can we remove it?

Package vulnerabilities

If I create a new apikana project and then run npm install I get many audit warnings:

npm WARN [email protected] requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.

audited 689 packages in 5.477s
found 283 vulnerabilities (158 low, 39 moderate, 85 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details

I think those warnings should be adressed.

Force empty cache necessary on Browser with embedded API viewer

When using Apikana Jar files with embedded browser, the browser does show always only the first one viewed. Any further API jar file started, does provide the same URL and therefore the browser shows just the cached former page.

Pressing Ctrll-F5 resolves the problem.

The corresponding webpage served by Apikana should be adapted to disable caching in order to force the browser to always load the current API definition instead.

Linked to https://jira.post.ch/browse/SDCISA-6387

Ranges

Is it possible to add ranges validation?
Writing as issue because couldn't find support.
I see that Swagger spec allows min. and max. values and I would like to have that feature.

Generating double slashes in APIPathBuilder

'/timetable/disruption/v1/areas/{planningAreaNumber}/alerts/{alertId}':
leads to

    public static final class AlertId extends Endpoint {
        private final String value;
        private AlertId(String alertId){ this.value = alertId; }
        public final String path() { return "/timetable/disruption/v1/areas/{planningAreaNumber}/alerts/" + "/" + value; }
    }

Fix Automated Travis Release

Automated release through travis seems not to work anymore (See: Travis Build).

We get:

npm ERR! publish Failed PUT 401
npm ERR! code E401
npm ERR! You must be logged in to publish packages. : apikana

Generated Schemas should implement schema draft 6 or newer

Apikana is currently generating json schemas based on draft version 4. We would like to use JsonSchema.Net to validate information against the provided schema. However, it does only support schema draft 6 or higher.

Would it be possible to support at least draft version 6 with apikana?
json-schema.org provides migration guides for the various version if any changes are needed.

Can't resolve reference from id

Since Apikana now creates schema draft7 it replaces the id with $id and now the validation fails when using references inside definitions that use $id. However, the JSON schema itself is actually valid, but it seems that ajv is not able to compile it properly.

Here is an example schema:

{
  type: "object",
  $id: "test",
  properties: {
    foo: {type: "integer"},
    bar: {type: "string"},
    "spec": {
        "$ref": "#/definitions/TopicSpecDescriptor",
        "description": "Topic specification."
    }
  },
  definitions: {
    "TopicSpecDescriptor": {
        "type": "object",
        "properties": {
            "partitions": {
              "$ref": "#/definitions/Int",
              "description": "Number of partitions.\nNormally 10.\nSee <a href=\"https://kafka.apache.org/documentation/#intro_concepts_and_terms\" target=”_blank”>Kafka concepts and terms</a>"
            }
        },
        "$id": "TopicSpecDescriptor"
    },
    "Int": {
        "type": "integer"
    }
  },
  required: ["foo"],
  additionalProperties: false
}

It validates without any error on JSON Schema Validator but when using ajv, it fails with this error:

  missingRef: 'TopicSpecDescriptor#/definitions/Int',
  missingSchema: 'TopicSpecDescriptor' 

If I remove the $id from the schema, it works as usual. But I don't think that's a good approach, especially since Apikana creates the JSON schema using $id now.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.