Code Monkey home page Code Monkey logo

spec-json-schemas's Introduction

npm npm

Overview

This is a mono repository, which provides all the JSON Schema documents for validating AsyncAPI documents.

Two types of schemas

This repository contains JSON Schema files for all the versions of AsyncAPI specification. There are two types of JSON Schema files, with and without $id feature. We need two versions of schemas because of the differences it tooling implementation of JSON Schema $ref and $id keywords. Some implementations treat $id by default as prefix reference for $ref and require it, therefore it is needed to properly correlate $ref and $id values. Unfortunately other tools do not understand $id values and fail dereferencing. This is why we need two different versions of schemas, with and without the $id.

Releases and pre-releases

This repository contains JSON Schema files for official AsyncAPI releases and also for release candidates. Before you decide to use a specific JSON Schema file in production, make sure a corresponding official release of AsyncAPI specification is produced, not a release candidate.

JSON Schema which describes a version of AsyncAPI specification that is not yet officially released is considered an unstable pre-release that can change anytime and is not considered to be a breaking-change.

If you want to make sure you only use stable schemas, you have to make sure that you use only certain schema versions, not all by default.

JSON Schema vs AsyncAPI specification

These JSON Schema files do not reflect 1:1 the specification and shouldn't be treated as specification itself but rather as a tool (e.g., for validation).

These JSON Schema files shouldn't be used as the only tool for validating AsyncAPI documents because some rules described in the AsyncAPI specification can't be described with JSON Schema.

Libraries

In addition, this repo provides JavaScript and Go modules that make it easier to access JSON Schema files through code. These packages provide access only to schemas with version larger or equal 2.0.0.

Custom Validation Needs

If you decide to validate AsyncAPI documents only with the JSON Schema files provided in this repo, your AsyncAPI documents will not be properly validated. It's recommended to use AsyncAPI JavaScript Parser that uses the AsyncAPI JSON Schema files for validation but also implements additional custom validations.

The following additional custom validations need to be provided for documents prior to 3.0.0:

  • Variables provided in the url property have a corresponding variable object defined and its example is correct.
  • operationIds are not duplicated in the document.
  • messageIds are not duplicated in the document.
  • Server security is declared properly and the name has a corresponding securitySchemes definition in components with the same name.
  • Server securitySchemes is an empty array when the security type requires it.
  • Parameters specified in the channel name have corresponding parameters object defined.
  • Channel names do not contain URL parameters.
  • All servers listed for a channel in the servers property are declared in the top-level servers object.
  • Tags specified in any object have no duplicates. Check must be done for: the top-level object, operations (publish and subscribe), operation traits, channels, messages, and message traits.

At the moment, the AsyncAPI JavaScript parser does not cover all validation cases yet. All test cases and parsers coverage can be found here

Installation

NodeJS

npm install @asyncapi/specs
// OR by Yarn
yarn add @asyncapi/specs

Go

go get github.com/asyncapi/spec-json-schemas/v2

Usage

NodeJS

Grab a specific AsyncAPI version:

const asyncapi = require('@asyncapi/specs');

console.log(asyncapi.schemas['2.0.0'])
console.log(asyncapi.schemasWithoutId['2.0.0'])
// Do something with the schema.

Get a list of supported versions:

const versions = require('@asyncapi/specs').schemas;

console.log(versions);
// Outputs object:
//
// {
//   '2.0.0': [Object],
//   '2.1.0': [Object],
//   ...
// }
console.log(Object.keys(versions));
// Outputs array:
//
// [
//   '2.0.0',
//   '2.1.0',
//   ...
// ]

Note The main export of the package provides only supported versions of AsyncAPI (newer or equal to 2.0.0). To use older ones (e.g. 1.2.0) please import an absolute path like @asyncapi/specs/schemas/1.2.0;

Go

Grab a specific AsyncAPI version:

import "github.com/asyncapi/spec_json_schemas/v4"

func Do() {
    schema, err := spec_json_schemas.Get("1.1.0")
    if err != nil {
        panic(err)
    }

    // Do something with the schema
}

Migration guidelines

If you are currently using version 2, check out migration guideline to version 3. If you are currently using version 3, check out migration guideline to version 4. If you are currently using version 4, check out migration guideline to version 5. If you are currently using version 5, check out migration guideline to version 6.

Repository structure

This is the current project structure explained:

  • ./definitions - contain all the individual schemas that will automatically be bundled together to provide the schemas in ./schemas.
  • ./examples - contain most individual definition examples that will automatically be bundled together to provide example for each definition in the schemas in ./schemas.
  • ./tools/bundler - is the tool that bundles all the individual schemas together.
  • ./schemas - contain all automatically bundled and complete schemas for each AsyncAPI version. These schemas should NOT be manually changed as they are automatically generated. Any changes should be done in ./definitions.
  • ./extensions - contains all the schemas of the extensions that will automatically be bundled to provide informations about extensions.

Schema Bundling

Changes should not be done manually to the schemas in ./schemas, but instead, be done in their individual definitions located in ./definitions.

These definitions are automatically bundled together on new releases through the npm script prepublishOnly, which ensures the project is built. This is where the bundler is called.

For example, for 2.2.0, the bundler starts with the asyncapi.json file and recursively goes through all references ($ref) to create the appropriate bundled version.

Creating a new version

1a Automated:

npm run startNewVersion --new-version=x.x.x

Where x.x.x is the new version you want to create.

1a Manual

The manual process of creating a new version is to:

  1. Duplicate the latest version (y.y.y) under definitions (so we have the correct base to make changes from).
  2. Rename the folder to the new version (x.x.x).
  3. Duplicate the latest version (y.y.y) under examples (so we have the correct base to make changes from).
  4. Rename the folder to the new version (x.x.x).
  5. Search and replace in the new duplicated folder for y.y.y and replace it with x.x.x.

2 Further steps

  1. Edit the index.js file adding a new line with the new version. I.e.:

    '2.6.0': require('./schemas/2.6.0.json'),
    '2.6.0-without-$id': require('./schemas/2.6.0-without-$id.json'),
  2. Edit the index.d.ts file adding a new line with the types for the new version. I.e.:

    '2.6.0': JSONSchema7;
    '2.6.0-without-$id': JSONSchema7;
  3. Edit the schemas/all.schema-store.json file adding a new entry under the oneOf keyword with the new version. Remember about adding -without-$id suffix which points to alternative generated schema without $ids. I.e.:

    {
       "allOf":[
          {
             "properties":{
                "asyncapi":{
                   "const":"2.6.0"
                }
             }
          },
          {
             "$ref":"http://asyncapi.com/schema-store/2.6.0-without-$id.json"
          }
       ]
    }

Handling breaking changes

Whenever a Breaking Change is introduced, the following steps should be taken in Go package:

  1. Edit go.mod file, and increase the version package suffix in the module name. For example, if the current version is v2.0.0, and you are releasing v3.0.0, the module name should be github.com/asyncapi/spec-json-schemas/v3.

SchemaStore compatibility testing

AsyncAPI JSON Schema is referenced in SchemaStore. In many IDEs, like VSCode, some extensions integrate with SchemaStore, like YAML. This way we enable autocompletion, validation and tooltips that help write AsyncAPI documents.

Whenever you make changes in AsyncAPI JSON Schema, you should always manually verify that the schema is still supported by YAML and that it will be able to fetch and dereference it.

  • Make sure you have VSCode and YAML extension
  • Once you make changes in schemas, bundle them with npm run bundle
  • Edit this sample AsyncAPI document. Add a new field, hover over some property to see if tooltip still shows up. This sample document contains the following line that assures YAML uses your local schema and not the one from SchemaStore. Make sure it points to the schema bundle that you generated and that contains your changes:
    # yaml-language-server: $schema=YOUR-PROJECTS-DIRECTORY/spec-json-schemas/schemas/2.6.0-without-$id.json

Extensions

Extensions are a way to extend AsyncAPI specification with fields that are not yet defined inside the specification. To add JSON schema of the extension in this repository, you need to first make sure it is added to the extension-catalog repository.

How to add schema of the extension

  1. All the extensions must be present in ./extensions folder.

  2. A proper folder structure must be followed to add the extensions.

  3. A new folder just as x extension must be added with proper versioning and schema file.

  4. All the schemas must be added in a file named schema.json just as one is defined for x extension.

  5. Extension schema should not be referenced directly in the definition of the object it extends. For example if you add an extension for info, your extension's schema should not be referenced from info.json but infoExtensions.json. If the object that you extend doesn't have a corresponding *Extensions.json file, you need to create one.

spec-json-schemas's People

Contributors

acethecreator avatar alequetzalli avatar animeshkumar923 avatar asyncapi-bot avatar baldinof avatar balogal avatar c-zekeri avatar char0n avatar codingtenshi avatar ctasada avatar dalelane avatar dependabot[bot] avatar derberg avatar devilkiller-ag avatar dindihub avatar fmvilas avatar gmin2 avatar gokerakc avatar greenrover avatar jonaslagoni avatar magicmatatjahu avatar mastdev avatar pakisan avatar rmelian avatar sambhavgupta0705 avatar smoya avatar vinibiavatti1 avatar waleedashraf avatar z5eyhs0uubejr0svmx2o avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spec-json-schemas's Issues

Incorrect validation of openIdConnect server security definition

How to Reproduce

Using the test document in the CLI exposed a bug when servers define the following security:

      - type: openIdConnect
        openIdConnectUrl: openIdConnectUrl
        scopes:
        - 'some:scope:1'
        - 'some:scope:2'

Full document:

asyncapi: 3.0.0
id: 'urn:example:com:smartylighting:streetlights:server'
info:
  title: AsyncAPI Sample App
  version: 1.0.1
  description: This is a sample app.
  termsOfService: 'https://asyncapi.com/terms/'
  contact:
    name: API Support
    url: 'https://www.asyncapi.com/support'
    email: [email protected]
  license:
    name: Apache 2.0
    url: 'https://www.apache.org/licenses/LICENSE-2.0.html'
  tags:
    - name: e-commerce
    - name: another-tag
      description: Description...
  externalDocs:
    description: Find more info here
    url: 'https://www.asyncapi.com'
defaultContentType: application/json
servers:
  default:
    host: 'api.streetlights.smartylighting.com:{port}'
    protocol: mqtt
    description: Test broker
    variables:
      port:
        description: Secure connection (TLS) is available through port 8883.
        default: '1883'
        enum:
          - '1883'
          - '8883'
    security:
      - $ref: '#/components/securitySchemes/apiKey'
      - type: oauth2
        flows:
          implicit:
            authorizationUrl: 'https://example.com/api/oauth/dialog'
            availableScopes:
              'write:pets': modify pets in your account
              'read:pets': read your pets
        scopes:
          - 'write:pets'
      - type: openIdConnect
        openIdConnectUrl: openIdConnectUrl
        scopes:
        - 'some:scope:1'
        - 'some:scope:2'
  production:
    host: 'api.streetlights.smartylighting.com:{port}'
    pathname: /some/path
    protocol: mqtt
    description: Test broker
    variables:
      port:
        description: Secure connection (TLS) is available through port 8883.
        default: '1883'
        enum:
          - '1883'
          - '8883'
    security:
      - $ref: '#/components/securitySchemes/apiKey'
  withProtocol:
    host: 'api.streetlights.smartylighting.com:{port}'
    pathname: /some/path
    protocol: mqtt
    description: Test broker
    variables:
      port:
        description: Secure connection (TLS) is available through port 8883.
        default: '1883'
        enum:
          - '1883'
          - '8883'
    security:
      - $ref: '#/components/securitySchemes/apiKey'
channels:
  'smartylighting/streetlights/1/0/event/{streetlightId}/lighting/measured':
    address: 'smartylighting/streetlights/1/0/event/{streetlightId}/lighting/measured'
    messages:
      lightMeasured.message:
        payload:
          type: object
    servers:
      - $ref: '#/servers/production'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
  'smartylighting/streetlights/1/0/action/{streetlightId}/turn/on':
    address: 'smartylighting/streetlights/1/0/action/{streetlightId}/turn/on'
    messages:
      publish.message:
        $ref: '#/components/messages/lightMeasured'
      subscribe.message.0:
        $ref: '#/components/messages/turnOnOff'
      customMessageId:
        messageId: customMessageId
        payload:
          type: object
      subscribe.message.2:
        payload:
          type: object
      subscribe.message.3:
        $ref: 'https://example.com/message'
    servers:
      - $ref: '#/servers/default'
      - $ref: '#/servers/production'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
  customChannelId:
    address: 'smartylighting/streetlights/1/0/action/{streetlightId}/turn/off'
    messages:
      turnOnOff.message:
        $ref: '#/components/messages/turnOnOff'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
    x-channelId: customChannelId
  'smartylighting/streetlights/1/0/action/{streetlightId}/dim':
    address: 'smartylighting/streetlights/1/0/action/{streetlightId}/dim'
    messages:
      dimLight.message:
        $ref: '#/components/messages/dimLight'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
operations:
  lightMeasured:
    action: receive
    channel:
      $ref: >-
        #/channels/smartylighting~1streetlights~11~10~1event~1{streetlightId}~1lighting~1measured
    messages:
      - $ref: >-
          #/channels/smartylighting~1streetlights~11~10~1event~1{streetlightId}~1lighting~1measured/messages/lightMeasured.message
  'smartylighting/streetlights/1/0/action/{streetlightId}/turn/on.publish':
    action: receive
    channel:
      $ref: >-
        #/channels/smartylighting~1streetlights~11~10~1action~1{streetlightId}~1turn~1on
    messages:
      - $ref: '#/components/messages/lightMeasured'
  'smartylighting/streetlights/1/0/action/{streetlightId}/turn/on.subscribe':
    action: send
    channel:
      $ref: >-
        #/channels/smartylighting~1streetlights~11~10~1action~1{streetlightId}~1turn~1on
    messages:
      - $ref: '#/components/messages/turnOnOff'
      - $ref: >-
          #/channels/smartylighting~1streetlights~11~10~1action~1{streetlightId}~1turn~1on/messages/customMessageId
      - $ref: >-
          #/channels/smartylighting~1streetlights~11~10~1action~1{streetlightId}~1turn~1on/messages/subscribe.message.2
      - $ref: 'https://example.com/message'
  turnOnOff:
    action: send
    channel:
      $ref: '#/channels/customChannelId'
    messages:
      - $ref: '#/components/messages/turnOnOff'
  dimLight:
    action: send
    channel:
      $ref: >-
        #/channels/smartylighting~1streetlights~11~10~1action~1{streetlightId}~1dim
    security:
      - type: oauth2
        flows:
          implicit:
            authorizationUrl: 'https://example.com/api/oauth/dialog'
            availableScopes:
              'write:pets': modify pets in your account
              'read:pets': read your pets
        scopes:
          - 'write:pets'
    messages:
      - $ref: '#/components/messages/dimLight'
components:
  messages:
    lightMeasured:
      summary: >-
        Inform about environmental lighting conditions for a particular
        streetlight.
      payload:
        $ref: '#/components/schemas/lightMeasuredPayload'
    turnOnOff:
      summary: Command a particular streetlight to turn the lights on or off.
      payload:
        $ref: '#/components/schemas/turnOnOffPayload'
    dimLight:
      summary: Command a particular streetlight to dim the lights.
      payload:
        $ref: '#/components/schemas/dimLightPayload'
  schemas:
    lightMeasuredPayload:
      type: object
      properties:
        lumens:
          type: integer
          minimum: 0
          description: Light intensity measured in lumens.
        sentAt:
          $ref: '#/components/schemas/sentAt'
    turnOnOffPayload:
      type: object
      properties:
        command:
          type: string
          enum:
            - 'on'
            - 'off'
          description: Whether to turn on or off the light.
        sentAt:
          $ref: '#/components/schemas/sentAt'
    dimLightPayload:
      type: object
      properties:
        percentage:
          type: integer
          description: Percentage to which the light should be dimmed to.
          minimum: 0
          maximum: 100
        sentAt:
          $ref: '#/components/schemas/sentAt'
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.
  securitySchemes:
    apiKey:
      type: apiKey
      in: user
      description: Provide your API key as the user and leave the password empty.
    flows:
      type: oauth2
      flows:
        implicit:
          authorizationUrl: 'https://example.com/api/oauth/dialog'
          availableScopes:
            'write:pets': modify pets in your account
            'read:pets': read your pets
    openIdConnect:
      type: openIdConnect
      openIdConnectUrl: openIdConnectUrl
    unusedFlows:
      type: oauth2
      flows:
        implicit:
          authorizationUrl: 'https://example.com/api/oauth/dialog'
          availableScopes:
            'write:pets': modify pets in your account
            'read:pets': read your pets
  parameters:
    streetlightId:
      description: The ID of the streetlight.

I have not checked whether it works for v2.

Expected behavior

Expected this to be a valid document.

[📑 Docs]: Write down all the custom validation rules that are not in JSON Schema

What Dev Docs changes are you proposing?

There is a need to write down what is not in JSON Schema and we wrote custom validation code for it here https://github.com/asyncapi/parser-js/blob/master/lib/customValidators.js so it is transparent and clear what happens if someone decides to validate AsyncAPI document by using directly the AsyncAPI JSON Schema document.

Even our Parser JS is not yet 100% accurate with validation, so https://asyncapi.github.io/tck/asyncapi-parser_js_detailed_report.html should also be treated as source of truth

Code of Conduct

  • I agree to follow this project's Code of Conduct

AsyncAPI spec v3 support in JSON Schema files

Reason/Context

This Issue is used to track changes needed to support AsyncAPI v3. As a code owner, please edit this list of TODO tasks in order to properly track the progress 🙂 Once this issue is closed it means that v3 is now fully supported in this library.

Although most changes are actually done here, because it's part of the release flow, there are still some outstanding issues.

Scope

  • Allow Multi-format in JSON Schema files #370
  • Release to production (master)

Maybe we should add some regex patterns for server host and pathname

in spec v3 we have new fields in Server Object:

  • host
  • pathname

in json schema we do not have any basic validation: https://github.com/asyncapi/spec-json-schemas/blob/next-major-spec/definitions/3.0.0/server.json#L15-L22

I think we should have at least some basic, that will fail if someone uses host the same way as it was with url, so localhost:3000/ws would be an error

JSON schema format keywords, might not be good, as hostname I think should be without port for example. I'm not 100% sure what formats from https://json-schema.org/understanding-json-schema/reference/string#built-in-formats we can reuse, thus suggesting patterns

Also in specification we are not so detailed and accurate, not pointing to standards like JSON Schema

cc @fmvilas that introduced a change in spec
cc @jonaslagoni release coordinator
cc @magicmatatjahu who I think introduced changes in json schemas
cc codeowners of the json schemas @dalelane @char0n @smoya

Enable reusability of schemas between versions

Reason/Context

There are specifically 2 schemas we wish to share among all versions.

  • avroSchema_v1
  • openapiSchema_3_0

To achieve this we need to do the following:

  1. Move the common schemas to a common definition folder
  2. Adapt the bundler (tools/bundler/index.js) to include these common definitions for all versions.

Support `security` field at Operation level

Reason/Context

Part of asyncapi/spec#584.

Description

Support security field containing an array of SecurityRequirement at Operation level.

As this is a good candidate to be in v2.3.0 release, the PR should be created from and targeting 2022-01-release branch.
Also 2.3.0.json should be the file where these changes should be added then.

As far as I see, the change we need to do is to add a new field called security into the operation definition. I think we can copy the one from the server found in https://github.com/asyncapi/spec-json-schemas/blob/2022-01-release/schemas/2.3.0.json#L192-L197

Security scheme object is not working as per docs

Describe the bug

I get Property name is not allowed.yaml-schema: AsyncAPI 3.0.0 schema.(0) when using name parameter in Security Scheme Object in v3 spec.

Screenshot 2023-11-08 at 4 04 20 PM

How to Reproduce

Link for document in Studio.

Expected behavior

As per docs we should be able to use the name parameter in the AsyncAPI document under Security Scheme Object.

[CI] Lint and validate JSON Schema docs

Reason/Context

In order to follow an standard, but also ensuring any change won't break the JSON Schema documents, I suggest we add a new lint + validate action in our GH CI workflows that run on each PR and block the PR if JSON Schema files are not valid or following our requirements.

Description

Consider adding a new script to the package.json scripts section, that lints and validate JSON Schema files. Something like npm run lint or similar. Also could be included in npm test call. In that way, developers could run that command before pushing code but also during the execution of CI tests.

Run the following CI tasks on the JSON Schema files on each PR. It doesn't mean all of the following should run in different actions since most of them could be done just with one tool like Spectral.

  • JSON linting
  • Validate JSON Schema files structure
  • [ ] Add some rules (Spectral?) for validating things like checking all definitions come with their examples and description fields. Any others are welcome as well. Not included for the bounty program

Detailed context

  • This repository holds JSON Schema documents, which are used for validating AsyncAPI documents (transformed to JSON). The parsers mainly use those JSON Schemas (parser-js, for instance) to validate AsyncAPI documents.
  • The JSON Schemas used by the parsers and any other tool that wants to validate AsyncAPI documents are located in the /schemas dir.
  • Those schemas result from bundling several smaller schemas, the ones located under /definitions. Each of those small schemas usually refers to the validation related to just one object from the AsyncAPI spec, I.e., Message Object, Channel Object, etc.
  • When people need to change those schemas, they do not modify them directly under /schemas but rather modify a particular (sometimes several of them) file under /definitions. That's done this way to help developers focus on what they want to change and not deal with a big file containing the whole JSON Schema doc for the entire specification.
Technical details that concern this issue:
  • CI scripts that run on every Pull Request are these.
  • The step that says Run linter is not about lining JSON Schemas but linting Javascript code. So, it has nothing to do with the work we want to do here.
  • The latest step it runs is bundling all smaller schemas found under /definitions into the bundled big ones located in /schemas.
  • We should validate those (/schemas), meaning our JSON Schema linting script should run after the generation of assets (the actual last step).

To illustrate the details above, here is a simple flow chart. Note that the script we should create is represented in the Validate schemas from /schemas step.

flowchart TD
    PR[PR] -->|Update schemas under /definitions| CI(CI / GH Actions)
    CI --> bundle(Bundle definitions into /schemas)
    bundle --> compile(Validate schemas from /schemas)
    subgraph The new action we should build
        compile -->result{Are schemas valid?}
        result -->|Yes| ok[OK - exit code 0]
        result -->|NO| ko[KO - print validation errors + exit code 1]
    end

We need to manually release `4.3.1`

Reason/Context

Few days ago 4.3.0 error happened.
I manually unpublished version 4.3 from npm but it was not removed, but marked as deprecated.

Problem is that if someone is using a project that depends on a project that depends on this package, they do not have influence over dependencies package.json, so if some project have ^4.1 in use, then 4.3 is anyway installed.

Temporary workaround is to use overrides in package.json to enforce 4.1

Description

Instead of making people use overrides feature from package.json, update docs and educate on request. @jonaslagoni have a good point that we could also release 4.3.1. More details: https://asyncapi.slack.com/archives/C34F2JV0U/p1683541443771049

Plan:

  • I can manually just release whatever is under 4.2.1 as 4.3.1. Will run release pipeline from my local.
  • Tag the same commit that has 4.2.1 as also 4.3.1
  • Publish release in GitHub with proper description

@char0n @smoya @dalelane @fmvilas any objections?

[Go] Look for a better approach to handle the module and major versions

Reason/Context

Following Go module major version suffixes creates complexity when releasing but also requires dependant libraries to manually update their go.mod and code in order to update to last major version (they need to change the import path in all the code).
More info https://go.dev/ref/mod#major-version-suffixes

Description

To find a solution, if possible that:

  • Allows releasing major versions without the need of adding the suffix at the end of the module name. I.e. the v4 suffix in github.com/asyncapi/spec-json-schemas/v4 module name.
  • Don't force dependant libraries to update all the imports of this module to replace the suffix to the last major version. That means the only change to do is in go.mod to update to the latest version.

Issues with the machine readable file for AsyncAPI 2.0.0

Describe the Issue
There are some discrepancies with the machine readable version of the AsyncAPI specification version 2.0.0.

  1. on line 571 additionalItems are defined when the schema type is object.
  2. When additionalProperties are indeed allowed on many of the schemas and should follow the patternProperties, how come it is always sat to false 🤔? I.e. on line 10
  3. On line 726 same as 1.
  4. On line 782 missing type: [object, array].

Traits can be defined as double arrays

Describe the bug

I just noticed that the operation traits https://github.com/asyncapi/spec-json-schemas/blob/master/definitions/2.5.0/operation.json#L13 can be defined with a double array:

{
      "type": "array",
      "items": {
        "oneOf": [
          {
            "$ref": "http://asyncapi.com/definitions/2.5.0/Reference.json"
          },
          {
            "$ref": "http://asyncapi.com/definitions/2.5.0/operationTrait.json"
          },
          {
            "type": "array",
            "items": [
              {
                "oneOf": [
                  {
                    "$ref": "http://asyncapi.com/definitions/2.5.0/Reference.json"
                  },
                  {
                    "$ref": "http://asyncapi.com/definitions/2.5.0/operationTrait.json"
                  }
                ]
              },
              {
                "type": "object",
                "additionalItems": true
              }
            ]
          }
        ]
      }
    }

I don't think this is the intended behavior as it allows you to define traits such as:

asyncapi: 2.5.0
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
channels:
  user/signedup:
    subscribe:
      traits: [[{summary: 'test'}]]

Go package

Reason/Context

I’m trying to give some 💛 to https://github.com/asyncapi/parser-go. I just wanted to add support for the latest spec versions (it only supports 2.0.0).
In order to get the JSON Schema files for each spec, so it supports all of them, we would need to follow a similar approach @asyncapi/specs` package does (built from this repository).

Description

I suggest we just add a .go file into this repository similar to index.js.
I have a POC here https://github.com/smoya/spec-json-schemas/blob/feat/go/spec-json-schemas.go

Split out definitions into separate files

Reason/Context

To make it more digestible and easier to pinpoint problems, I think it would make sense to split each definition into separate files.

Description

That said, I also see benefits in bundling all these separate definitions into one file. This way it remains easy to use in other settings, where you don't need all the separate files.

Cleaning up how sub schemas are defined with references

Reason/Context

After #128 it would be great to have a consistent way of defining how objects and references are defined.

In my opionion it would be better if for example a Message Object JSON Schema file, ONLY contained the information about how a message can be defined, and not that messages can be defined as both a Reference and a Message Object.

Currently, we defined a Message Object as being:

{
  "oneOf": [
    {
      "$ref": "http://asyncapi.com/definitions/2.3.0/Reference.json"
    },
    {
      ...
    }
  ],
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://asyncapi.com/definitions/2.3.0/message.json"
}

However, I feel like a Message Object should only be defined as is (without reference)

{
  ...,
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://asyncapi.com/definitions/2.3.0/message.json"
}

While the definition of Messages should contain the reference:

{
  "type": "object",
  "additionalProperties": {
    "oneOf": [
      {
        "$ref": "http://asyncapi.com/definitions/2.3.0/Reference.json"
      },
      {
        "$ref": "http://asyncapi.com/definitions/2.3.0/message.json"  
      }
    ]
  },
  "description": "JSON objects describing the messages being consumed and produced by the API.",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://asyncapi.com/definitions/2.3.0/messages.json"
}

Need for urgent changes in GitHub Actions automation

This issue defines a list of tasks that need to be performed in this repo to make sure it's ci/cd automation works long term without any issues.

It is up to maintainers to decide if it must be addressed in one or multiple PRs.

Below are 3 different sections describing 3 different important ci/cd changes.

IMPORTANT-START
For GitHub workflows that contain This workflow is centrally managed in https://github.com/asyncapi/.github/ you do not have to perform any work. These workflows were already updated through the update in .github. The only exception is the workflows related to nodejs release. More details in Upgrade Release pipeline - in case of nodejs projects section
IMPORTANT-END

Deprecation of way data is shared between steps

Every single GitHub Action workflow that has echo "::set-output name={name}::{value}" need to be updated to follow echo "{name}={value}" >> $GITHUB_OUTPUT

We do not yet know when set-output will stop working. Previous disable date was 31.05 but now then say community needs more time.

For more details read official article from GitHub

Deprecation of node12

2nd bullet point is still relevant for you even if your projects in not nodejs project

  • Every single workflow that uses setup-node action needs an update to follow v3 version of this action, and make sure minimum node 14 is used
  • Now this part is more complex. Problem with node12 is that node-based GitHub Actions were using it in majority as a runtime environment. Look for example at this action.yaml file for setup-node action v2. So the job that you have to do is go through all the workflows, and verify every single action that you use, make sure you are using the latest version that is not based on node12. I already did review a lot of actions as part of this PR so maybe you will find some actions there and can copy from me. For example action/checkout needs to be updated to v3.

Node12 end of support in action is probably September 27th.

For more details read official article from GitHub

Upgrade Release pipeline - in case of nodejs projects

ignore this section if your project is not nodejs project

You have 2 options. You can:

A. choose to switch to new release pipeline using instruction from asyncapi/.github#205

B. stay with old release pipeline, and manually update GitHub workflows and actions used in it, you can inspire a lot from this PR asyncapi/.github#226

I definitely recommend going with A

Workflows related to release:

  • .github/workflows/if-nodejs-release.yml
  • .github/workflows/if-nodejs-version-bump.yml
  • .github/workflows/bump.yml

Change the way schemas are exported

Part of asyncapi/parser-js#481

This change, once ported to next-major branch, will make https://github.com/asyncapi/parser-js/blob/next-major/src/schema-parser/asyncapi-schema-parser.ts#L38-L53 to provide wrong types.

TBH is not an issue with this PR per se. The way we are exporting the schemas is not helping. I think we could introduce the following separation in next-major branch:

declare const _exports: {
    'schemas': {
        // ...
    },
    'schemas-without-$id': {
        // ...
    }
}

Do you think it makes sense? If it does, +1 to merge this and start doing those changes in next-major.

cc @magicmatatjahu @jonaslagoni

Originally posted by @smoya in #298 (comment)

Add support for messageId

Reason/Context

This is a follow-up issue coming from asyncapi/spec#458.

Description

As per the RFC, we need to add support for messageId at message level. This issue helps tracking progress on it.

Note: since the feature is not considered a breaking change, we could include it either on 2.3.0 or 3.0.0, depending on how "fast" we move on.

cc @WaleedAshraf

validate extension with json schema

This issue is a part of AsyncAPImentorship program #78

Main Objective

As part of the Mentorship program we need to validate the extensions with json schema.
A folder named extensions would be added just as bindings are present.

Description

#280
In brief, An extension enables the description of functionality or protocol related features. Everyone can create their own extensions to give users more freedom on the spec.

We need to have a JSON Schema validation. In fact, to enable validation in the parser, we need JSON Schema for AsyncAPI to contain JSON Schemas of individual extensions.

The final objective is to have a catalog of extensions that users can browse and use. Then, as time goes by, we enhance the catalog according to the demands / needs.

cc: @derberg

Message object allows nested `oneOf`

Describe the bug

It is possible to define a message such as:

      message:
        oneOf:
          - name: LightMeasured
            payload:
              type: string
          - oneOf: 
            - name: LightMeasured2
              payload:
                type: string

and having the schema valid, this should not be the case.

How to Reproduce

asyncapi: 2.6.0
info:
  title: Streetlights API
  version: 1.0.0
  description: |
    The Smartylighting Streetlights API allows you
    to remotely manage the city lights.
  license:
    name: Apache 2.0
    url: 'https://www.apache.org/licenses/LICENSE-2.0'
servers:
  mosquitto:
    url: 'mqtt://test.mosquitto.org'
    protocol: mqtt
channels:
  light/measured:
    publish:
      summary: >-
        Inform about environmental lighting conditions for a particular
        streetlight.
      operationId: onLightMeasured
      message:
        oneOf:
          - name: LightMeasured
            payload:
              type: string
          - oneOf: 
            - name: LightMeasured2
              payload:
                type: string

Expected behavior

I was expected the validation of said AsyncAPI file to be invalid instead of valid.

Extensions Catalog

Reason/Context

As part of the "MVP integration of extensions catalog with AsyncAPI tools to make extension catalog useful" mentorhsip program, detailed here asyncapi/extensions-catalog#78. We would like to add in this repo a part that concerns the description of extensions. In the same way as the bindings and spec.

Description

In brief, An extension enables the description of functionality or protocol related features. Everyone can create their own extensions to give users more freedom on the spec.

We need to have a JSON Schema validation. In fact, to enable validation in the parser, we need JSON Schema for AsyncAPI to contain JSON Schemas of individual extensions.

The final objective is to have a catalog of extensions that users can browse and use. Then, as time goes by, we enhance the catalog according to the demands / needs.

[JSON Schema Store] IDEs cannot determine the version of the JSON Schema to use

Describe the bug

IDEs, like VSCode, cannot determine the version of the JSON Schema file to use based on the AsyncAPI version in use.

2022-06-27 at 09 48 32

How to Reproduce

Expected behavior

IDEs should pick up the proper JSON Schema file based on the version selected on the AsyncAPI file.

The solution can be found in SchemaStore/schemastore#2229 (comment)

$ref do not work with `bindings` in channel in v3

issue might be with any bindings, not just channel, I did not check

spec says that in channels I can also have $ref

Screenshot 2023-10-31 at 14 17 15

which makes sense as I want to have some reusable stuff in components. channelBindings

I would expect below example to work (studio direct link - btw I love new share button in studio:

asyncapi: '3.0.0'
info:
  title: MySlackBot API
  version: '1.0.0'
  description:  |
    The MySlackBot App manages popular messages in a workspace by monitoring message reaction data from Slack's Event API.
    
servers:
  ws:
    host: wss://wss-primary.slack.com/
    protocol: wss
    description: "Websocket URL generated to communicate with Slack"
channels:
  root:
    address: /
    messages:
      reactionListener: {}
      helloListener: {}
    bindings:
      $ref: "#/components/channelBindings/connectionQuery"
      
operations:
  helloListenerOperation:
    action: receive
    channel:
      $ref: "#/channels/root"
    messages: 
      - $ref: "#/channels/root/messages/helloListener"
  reactionListenerOperation:
    action: receive
    channel: 
      $ref: "#/channels/root"
    messages: 
      - $ref: "#/channels/root/messages/reactionListener"

components:
  channelBindings:
    connectionQuery:
      ws:
        query: {}

but this says is not allowed

bindings:
      $ref: "#/components/channelBindings/connectionQuery"

After so many years, I still do not feel like JSON Schema expert so can you confirm, but I think in here https://github.com/asyncapi/spec-json-schemas/blob/next-major-spec/definitions/3.0.0/channel.json#L65 we should instead have something like

"bindings": {
      "oneOf": [
        {
          "$ref": "http://asyncapi.com/definitions/3.0.0/Reference.json"
        },
        {
          "$ref": "http://asyncapi.com/definitions/3.0.0/channelBindingsObject.json"
        }
      ]

thoughts?

Provide JSON Schema draft 7 file as part of schemas

Reason/Context

JSON Schema Draft 7 is the JSON Schema version AsyncAPI is compatible with.
All our schemas are written with such dialect, ergo we specify it through the $schema field.
We also make references to that same schema when referencing through $ref.

As you can notice, we strongly depend on such JSON Schema schema.
I'm suggesting we include it as part of our schemas collection so any tool can directly load all schemas, including the JSON Schema Draft 7 on their Schema registries so they don't need to fetch it externally.

A recent real use case can be found asyncapi/studio#159, where the JSON Schema Draft 7 should be provided to Monaco editor YAML plugin in order to avoid fetching it through a HTTP Request.

The plan will also include one JSON Schema schema file per version we require, meaning if we ever adopt a newer JSON Schema draft, we will include it as well here.

Description

Add a new file into /schemas called json-schema-draft-07.json with the exact content of http://json-schema.org/draft-07/schema.

Migrate package-lock.json file to version 2

Reason/Context

Parser-JS requires now Node.js 14 and npm7 or higher.
This means package-lock.json version should be always set to 2.

Description

In order to unify development requirements, I suggest all repositories to stick with that requirements. In order to do that, the following is needed:

  • Clarify this requirement under Development section on this repository README file.
  • To update package-lock.json to version 2. This is automatically done when running npm install if the npm version used is 7 or higher.

Related:

Example definition for message object and message trait should be the same

Imho https://github.com/asyncapi/spec-json-schemas/blob/next-major-spec/definitions/3.0.0/messageObject.json#L73 should be replicated in https://github.com/asyncapi/spec-json-schemas/blob/next-major-spec/definitions/3.0.0/messageTrait.json#L70

cause if somebody will be allowed anything with examples in trait, then trait merging will never work

probably best solution would be to introduce separate reusable MessageExamples.json and reuse it in messageObject.json and messageTrait.json with $ref

Component messages can contain oneOf

Describe the bug

According to the spec, a message in the component section should not be able to be defined with oneOf, but at the moment the following is valid:

components:
  messages:
    myMessage:
      oneOf:
        - name: LightMeasured
          payload:
            type: string

Expected behavior

Expected a validator to mark this as invalid input.

Schema for v2.0.0 does not allow to extend the components object with specification extensions

Describe the bug

The schema for v2.0.0 (and all older versions) does not allow to extend the components object with specification extensions. This is inconsistent with the specification which states that the components object can be extended with specification extensions.

How to Reproduce

Validating the following exemplary AsyncAPI document against the v2.0.0 schema with any JSON schema validation library results in an error like Property 'x-extension' has not been defined and the schema does not allow additional properties or similar:

{
    "asyncapi": "2.0.0",
    "info": {"version": "1.0", "title": "Example API"},
    "channels": {},
    "components": {
        "x-extension": "xxx"
    }
}

Expected behavior

Above AsyncAPI document should be validated without errors.

drop asyncapi json schemas prior 2.0 and release 4.0

we still expose AsyncAPI 1.0, 1.1 and 1.2 but we do not really support it anymore.

we should remove it and release new major

these old schemas use draft 4 as base. The new ones use draft 7. So in parser we are forced to point to draft 4 as meta schema for ajv. So we do something in the parser that do not make much sense, as parser do not support asyncapi prior 2.0

thoughts?

cc approvers @fmvilas @dalelane

Cannot build index.d.ts file due to an invalid export syntax

Describe the bug

Since the option esModuleInterop: true is not set to the project, the file index.d.ts cannot be compiled. This happens because the module definition of the file uses an invalid export syntax, when it should use the export= syntax type. This is described better on this documentation: Module.d.ts

Note that using export default in your .d.ts files requires [esModuleInterop: true](https://www.typescriptlang.org/tsconfig#esModuleInterop) to work. If you can’t have esModuleInterop: true in your project, such as when you’re submitting a PR to Definitely Typed, you’ll have to use the export= syntax instead

File path: https://github.com/asyncapi/spec-json-schemas/blob/master/index.d.ts

How to Reproduce

npm run build:prod (tested in local environment only)

Link to GitHub repository with project that has issues:
https://github.com/asyncapi/server-api

Expected behavior

After running the command, it should finish without any build errors.

Rename this repo?

Should we rename this repo to schemas or similar? It will not be only for Node.js but for other languages too.

Vscode unable to resolve parts of the schema

Describe the bug

According to redhat-developer/yaml-language-server#585 and microsoft/vscode-json-languageservice#123 the use of hashes to reference eternal schemas is not supported.
$ref '/definitions/schemaArray' in 'http://asyncapi.com/schema-store/2.4.0.json' can not be resolved.

How to Reproduce

Create a new asyncapi.json or asyncapi.yaml file in vscode with the yaml extention installed.

image

Expected behavior

The schema should resolve by the language servers used by the extention.

[Security] Workflow stale-issues-prs.yml is using vulnerable action actions/stale

The workflow stale-issues-prs.yml is referencing action actions/stale using references v1.1.0. However this reference is missing the commit af4072615903a8b031f986d25b1ae3bf45ec44d4 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

remove js and go package and probably move json schemas should stay alone in the repo

#124 introduced symetric releases for the package. So basically whenever there is a change in Go package, we will also release JS package in all the different projects that depend on it. Cause a lot of noise with "empty" functionality.

  1. Go package should have its own release cycle. Symetric releases are cool but for cases like in react-component where we release web-component and other things at the same time
  2. I know some time ago we took JSON Schema files out from spec repo and moved it here to have one project with JSON Schemas and projects for different languages. But I admit, that in this discussion I never took into consideration release process and how strange things will work here. IMHO the best would be to leave JSON Schemas in this repo alone, then JS code should go to js-json-schemas and Go should go to something like go-json-schemas. Once JSON Schema is released with some update, there should be a workflow that pushes updated json schema to JS, Go and other projects and release is triggered there automatically

Related conversation: #118

Cleanup GitHub Actions workflows

Context

  1. We want to make sure we have healthy community and people respec each other in GitHub. We want to monitor all comments in every repository and analyze sentiments to identify negative ones and get proper notification. Idea is to use GitHub Actions for it, to check sentiments with Google Natural Language API and post negative comments to Slack for review.

  2. We manage community files in global .github repo, and we have GitHub Funding config file there too. As a result we need to cleanup all the repos to remove .github/FUNDING.yml

  3. Start supporting new GitHub pull request event that allows gives access to GITHUB_TOKEN with write access. Fix welcome contributors workflow is needed.

  4. We want to automatically merge Pull Requests that are submitted by Dependanbot. New workflow needs to be added, or if already existins it needs to be modified

Description

  1. Add this workflow file called sentiment-analysis.yml to .github/workflows (create it if it doesn't exist yet) directory: https://gist.github.com/derberg/ab362e4b37f542e7e1813e67b7cb11ee
    Proper secrets are already added to GitHub organization so it is as simple as adding above file to workflows dir.
  2. Remove .github/FUNDING.yml if it exists
  3. If not done already rename pull_request from line 4 to pull_request_target in .github/workflows/welcome-first-time-contrib.yml file
  4. Create new file called automerge.yml with the following content: https://gist.github.com/derberg/024814a26959d54f683e7bd68d68f007
    If the repository already has a file called automerge-release-pr-bump.yml than rename it and adjust if statements to check github.actor == 'asyncapi-bot' || github.actor == 'dependabot[bot]' || github.actor == 'dependabot-preview[bot]' instead of github.actor == 'asyncapi-bot'

Cannot define channel's parameter as reference

Describe the bug

As we can see in the Studio:

image

image

In the channel's parameters we cannot make references.

How to Reproduce

Open https://studio.asyncapi.com/?url=https://raw.githubusercontent.com/asyncapi/spec/master/examples/streetlights-kafka.yml and convert to the JSON - right corner of the editor -> Convert to JSON and you should see yellow marker under $ref used in any channel's parameter (in YAML format you won't see error due to the asyncapi/studio#509 issue).

Expected behavior

As a user I can make reference to the channel's parameter without "referencing" errors.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.