Code Monkey home page Code Monkey logo

hawaii's Introduction

Hawaii Nuget

A dotnet CLI tool to generate type-safe F# and Fable clients from OpenAPI/Swagger/OData services.

Features

  • Supports any OpenApi/Swagger schema in form of JSON or Yaml, compatible with those that can be read by OpenAPI.NET
  • Supports OData services (see example below), made possible by OpenAPI.NET.OData which translates the OData model into an OpenApi document
  • Generates clients for F# on dotnet or for Fable in the browser
  • Automatically handles JSON deserialization into schema types
  • Automatically handles mixed responses from end points that return binary or typed JSON responses
  • Generates discriminated union types to describe the possible responses of each endpoint
  • Generates full F# projects, including their required dependencies and targeting netstandard2.0

Install

dotnet tool install -g hawaii

Configuration

Create a configuration file called hawaii.json with the following shape:

{
    "schema": <schema>,
    "project": <project>,
    "output": <output>,
    ["target"]: <"fsharp" | "fable">
    ["synchronous"]: <true | false>,
    ["asyncReturnType"]: <"async" | "task">,
    ["resolveReferences"]: <true | false>,
    ["emptyDefinitions"]: <"ignore" | "free-form">,
    ["overrideSchema"]: <JSON schema subset>,
    ["filterTags"]: <list of tags>
}

Where

  • <schema> is a URL to the OpenApi/Swagger/OData location, whether that is an external URL or a relative file path. In case of OData services, the external URL has to end with $metadata which points to the Edm model of that service (see TripPinService example below) or it can be a local .xml file that contains the schema
  • <project> is the name of the project that will get generated
  • <output> is a relative path to the output directory where the project will be generated. (Note: this directory is deleted and re-generated when you run hawaii)
  • <synchronous> is an optional flag that determines whether hawaii should generate client methods that run http requests synchronously. This is useful when used inside console applications. (set to false by default)
  • <target> specifies whether hawaii should generate a client for F# on dotnet (default) or a Fable client
  • <asyncReturnType> is an option to determine whether hawaii should generate client methods that return Async<'T> when set to "async" (default) or Task<'T> when set to "task" (this option is irrelevant when the synchronous option is set to true)
  • <resolveReferences> determines whether hawaii will attempt to resolve external references via schema pre-processing. This is set to false by default but sometimes an OpenApi schema is scattered into multiple schemas across a repository and this might help with the resolution.
  • <emptyDefintions> determines what hawaii should do when encountering a global type definition without schema information. When set to "ignore" (default) hawaii will generate the global type. However, sometimes these global types are still referenced from other types or definitions, in which case the setting this option to "free-form" will generate a type abbreviation for the empty schema equal to a free form object (JToken when targetting F# or obj when targetting Fable)
  • <overrideSchema> Allows you to override the resolved schema either to add more information (such as a missing operation ID) or correct the types when you know better (see below)
  • <filterTags> Allows to filter which operations will be included based on their OpenAPI tags. Useful when generating the full schema isn't possible or isn't practical. To see what tags are available, use hawaii --show-tags

Example (PetStore Schema)

Here is an example configuration for the pet store API:

{
    "schema": "https://petstore3.swagger.io/api/v3/openapi.json",
    "output": "./output",
    "project": "PetStore",
    "synchronous": true
}

After you have the configuration ready, run hawaii in the directory where you have the hawaii.json file:

hawaii

You can also tell hawaii where to find the configuration file if it wasn't named hawaii.json, for example

hawaii --config ./petstore-hawaii.json

Using the generated project

Once hawaii has finished running, you find a fully generated F# project inside of the <output> directory. This project can be referenced from your application so you can start using it.

You can reference the project like this from your app like this:

<ItemGroup>
  <ProjectReference Include="..\path\to\output\PetStore.fsproj" />
</ItemGroup>

Then from your code:

open System
open System.Net.Http
open PetStore
open PetStore.Types

let petStoreUri = Uri "https://petstore3.swagger.io/api/v3"
let httpClient = new HttpClient(BaseAddress=petStoreUri)
let petStore = PetStoreClient(httpClient)

let availablePets() =
    let status = PetStatus.Available.Format()
    match petStore.findPetsByStatus(status) with
    | FindPetsByStatus.OK pets -> for pet in pets do printfn $"{pet.name}"
    | FindPetsByStatus.BadRequest -> printfn "Bad request"

availablePets()

// inventory : Map<string, int>
let (GetInventory.OK(inventory)) = petStore.getInventory()

for (status, quantity) in Map.toList inventory do
    printfn $"There are {quantity} pet(s) {status}"

Notice that you have to provide your own HttpClient to the PetStoreClient and set the BaseAddress to the base path of the service.

Example with OData (TripPinService schema)

{
  "schema": "https://services.odata.org/V4/(S(s3lb035ptje4a1j0bvkmqqa0))/TripPinServiceRW/$metadata",
  "project": "TripPinService",
  "output": "./output"
}

Generate OpenAPI specs from the OData schemas

Sometimes you want to see how Hawaii generated a client from an OData schema.

You use the following command to generate the intermediate OpenAPI specs file in the form of JSON.

Then you can inspect it but also modify then use it as your <schema> when you need to make corrections to the generated client

hawaii --from-odata-schema {schema} --output {output}

where

  • {schema} is either a URL to an external OData schema or local .xml file which the contents of the schema
  • {output} is a relative file path where the translated OpenAPI specs will be written

Example of such command

hawaii --from-odata-schema "https://services.odata.org/V4/(S(s3lb035ptje4a1j0bvkmqqa0))/TripPinServiceRW/$metadata" --output ./TripPin.json

Sample Applications

  • Minimal PetStore - A simple console application showing how to use the PetStore API client generated by Hawaii
  • Feliz PetStore - A Feliz application that shows how to use the PetStore API client generated by Hawaii when targeting Fable
  • OData TripPin Service - This application demonstrates a console application that uses a client for the TripPin OData service generated by Hawaii

Version

You can ask hawaii which version it is currently on:

hawaii --version

No Logo

If you don't want the logo to show up in your CI or local machine, add --no-logo as the last parameter

hawaii --no-logo
hawaii --config ./hawaii.json --no-logo

Advanced - Overriding The Schema

OpenAPI schemas can be very loose and not always typed. Sometimes they will be missing operation IDs on certain paths. Although Hawaii will attempt to derive valid operation IDs from the path, name collisions can sometimes happen. Hawaii provides the overrideSchema option to allow you to "fix" the source schema or add more information when its missing.

Here is an example for how you can override operation IDs for certain paths

{
  "overrideSchema": {
    "paths": {
      "/consumer/v1/services/{id}/allocations": {
        "get": {
          "operationId": "getAllocationsForCustomerByServiceId"
        }
      },
      "/consumer/v1/services/allocations/{id}": {
        "get": {
            "operationId": "getAllocationIdFromCustomerServices"
        }
      }
    }
  }
}

The overrideSchema property basically takes a subset of another schema and merges it with the source schema.

You can go a step further by overriding the return types of certain responses. The following example shows how you can get the raw text output from the default response of a path instead of getting a typed response:

{
  "overrideSchema": {
    "paths": {
      "/bin/querybuilder.json": {
        "get": {
          "responses": {
            "default": {
              "content": {
                "application/json": {
                  "schema": {
                    "type": "string"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Limitations

These are the very early days of Hawaii as a tool to generate F# clients and there are some known limitations and rough edges that I will be working on:

  • anyOf/oneOf not supported unless they contain a single element, in which case they are reduced away

You can watch the live coding sessions as a playlist published on YouTube here

Running integration tests

cd ./build
# run hawaii against multiple config permutations
dotnet run -- generate-and-build
# run hawaii against 10 schemas
dotnet run -- integration
# run hawaii agains the first {n} schemas out of ~2000 and see the progress
dotnet run -- rate {n}

hawaii's People

Contributors

francotiveron avatar numpsy avatar randrag avatar zaid-ajaj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

hawaii's Issues

Implement better handling of mixed responses for JSON and binary

Sometimes an operation will return either byte[] in the OK case or will return typed objects (formatted as JSON) in the other cases:

type DownloadError = { error: string }

[<RequireQualifiedAccess>]
type DownloadFile =
    | OK of payload: byte []
    | Unauthorized of payload: DownloadError
    | BadRequest of payload: DownloadError

Right now, the client implementation looks like this

type FilesClient(httpClient: HttpClient) = 
      member this.downloadFile(fileName: string) =
        let requestParts =
            [ RequestPart.query ("fileName", fileName) ]

        let (status, content) =
            OpenApiHttp.post httpClient "/api/download" requestParts

        if status = HttpStatusCode.OK then
            DownloadFile.OK(System.Text.Encoding.UTF8.GetBytes content)
        else if status = HttpStatusCode.Unauthorized then
            DownloadFile.Unauthorized(Serializer.deserialize content)
        else
            DownloadFile.BadRequest(Serializer.deserialize content)

Where the HTTP content is string at first but using System.Text.Encoding.UTF8.GetBytes to convert it back byte[]
This is not ideal, instead it should start with byte[] in default case and only be converted to string for the JSON deserialization

type FilesClient(httpClient: HttpClient) = 
      member this.downloadFile(fileName: string) =
        let requestParts =
            [ RequestPart.query ("fileName", fileName) ]

        let (status, contentBinary) =
            OpenApiHttp.postBinary httpClient "/api/download" requestParts

        if status = HttpStatusCode.OK then
            DownloadFile.OK contentBinary
        else if status = HttpStatusCode.Unauthorized then
            let content = System.Encoding.UTF8.GetString contentBinary
            DownloadFile.Unauthorized(Serializer.deserialize content)
        else
            let content = System.Encoding.UTF8.GetString contentBinary
            DownloadFile.BadRequest(Serializer.deserialize content)

To implement this, we need functions in the HTTP module to return status code * byte[] instead of status code * string and extend the decision tree implementation to convert the bytes into string before deserialization

Generate proper payload types from loose request body objects

cc @JonCanning

It seems that there is a problem with generating payload types for request JSON bodies, right now they are defaulting to payload: string which is incorrect. For example:

type PodiotsuiteClient(httpClient: HttpClient) =
    ///<summary>
    ///Takes username and password and if it finds match in back-end it returns an object with the user data and token used to authorize the login.
    ///</summary>
    member this.PostAuthToken(payload: string) =
        async {
            let requestParts = [ RequestPart.jsonContent payload ]
            let! (status, content) = OpenApiHttp.postAsync httpClient "/auth/token" requestParts
            if status = HttpStatusCode.OK then
                return PostAuthToken.OK(Serializer.deserialize content)
            else if status = HttpStatusCode.BadRequest then
                return PostAuthToken.BadRequest(Serializer.deserialize content)
            else if status = HttpStatusCode.Unauthorized then
                return PostAuthToken.Unauthorized(Serializer.deserialize content)
            else
                return PostAuthToken.InternalServerError(Serializer.deserialize content)
        }

It should be

type PostAuthTokenPayload = 
   { username: string;
     password: string }

type PodiotsuiteClient(httpClient: HttpClient) =
    ///<summary>
    ///Takes username and password and if it finds match in back-end it returns an object with the user data and token used to authorize the login.
    ///</summary>
    member this.PostAuthToken(payload: PostAuthTokenPayload) =
        async {
            let requestParts = [ RequestPart.jsonContent payload ]
            let! (status, content) = OpenApiHttp.postAsync httpClient "/auth/token" requestParts
            if status = HttpStatusCode.OK then
                return PostAuthToken.OK(Serializer.deserialize content)
            else if status = HttpStatusCode.BadRequest then
                return PostAuthToken.BadRequest(Serializer.deserialize content)
            else if status = HttpStatusCode.Unauthorized then
                return PostAuthToken.Unauthorized(Serializer.deserialize content)
            else
                return PostAuthToken.InternalServerError(Serializer.deserialize content)
        }

Originally posted by @Zaid-Ajaj in #12 (comment)

Any way to incorporate binary streams?

I am dealing with a response spec like this

"responses": {
          "200": {
            "description": "Retrieved media content",
            "content": {
              "application/octet-stream": {
                "schema": {
                  "type": "string",
                  "format": "binary"
                }
              }
            }
          },
          "default": {
            "$ref": "#/components/responses/error"
          }
        }

The return data can be very large, far larger than can fit into memory.
In this case, I have no choice but to consume it as a stream.

For content type application/octet-stream would you consider returning as a stream?

CLI problems

noticed some issues with the CLI

--version reports the wrong number

I tried to use --show-tags with --config, which did not work but I see now that it expects a file path without a --config keyword. Would be good if there was a --help detailing the interface in full.

On a related note, would you be open to porting the CLI to Argu? Gives you --help for free and makes it easy to conform to the style of other F# tools such as paket. If yes, I can work on a PR this weekend.

API:s requiring credentials fail

I have an API that requires a credentials cookie.

So when the OpenApiHttp.fs does this:

let sendAsync
    (method: HttpMethod)
    (basePath: string)
    (path: string)
    (extraHeaders: Header list)
    (parts: RequestPart list)
    : Async<int * string> =
    async {
        let requestPath = applyPathParts path parts

        let requestPathWithQuery =
            applyQueryStringParameters requestPath parts

        let fullPath =
            combineBasePath basePath requestPathWithQuery

        let! response =
            Http.request fullPath
            |> Http.method method
            |> applyJsonRequestBody parts
            |> applyMultipartFormData parts
            |> applyUrlEncodedFormData parts
            |> Http.headers extraHeaders
            |> Http.send

        let status = response.statusCode
        let content = response.responseText
        return status, content
    }

It doesn't include the credentials needed.
Maybe it can be fixed using Fable.SimpleHttp.Http.withCredentials? Like this:

let sendAsync
    (method: HttpMethod)
    (basePath: string)
    (path: string)
    (extraHeaders: Header list)
    (parts: RequestPart list)
    : Async<int * string> =
    async {
        let requestPath = applyPathParts path parts

        let requestPathWithQuery =
            applyQueryStringParameters requestPath parts

        let fullPath =
            combineBasePath basePath requestPathWithQuery

        let! response =
            Http.request fullPath
            |> Http.method method
            |> applyJsonRequestBody parts
            |> applyMultipartFormData parts
            |> applyUrlEncodedFormData parts
            |> Http.headers extraHeaders
            |> Http.withCredentials true // <------------- added this
            |> Http.send

        let status = response.statusCode
        let content = response.responseText
        return status, content
    }

Hawaii throws: Unhandled exception. Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: D. Path '', line 0, position 0.

Hi @Zaid-Ajaj, I was trying out this awesome project to create a client for Stripe (using this: https://github.com/stripe/openapi/blob/master/openapi/spec3.yaml), which has an OpenApi spec in JSON and YAML. However, either way I try it, hawaii crashes with the following exception:

Unhandled exception. Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: D. Path '', line 0, position 0.
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings)
   at Newtonsoft.Json.Linq.JObject.Parse(String json, JsonLoadSettings settings)
   at Newtonsoft.Json.Linq.JObject.Parse(String json)
   at Program.getSchema(String schema, FSharpOption`1 overrideSchema) in /Users/zaid/projects/Hawaii/src/Program.fs:line 290
   at Program.runConfig(String filePath) in /Users/zaid/projects/Hawaii/src/Program.fs:line 2767
   at Program.main(String[] argv) in /Users/zaid/projects/Hawaii/src/Program.fs:line 2920

This exception appears to incorrectly render the path/file name that causes the error, and "line 0, pos 0" suggests something odd is going on.

Tbh, I'm fully aware that this is a massive API from Stripe, so I wouldn't entirely expect it to go as smoothly as any smaller API. Though I also tried it with some smaller APIs and still got the same error, so maybe something else is the matter. To that end, this is the config I was using:

{
  "schema": "testapi.json",        // tried with stripe's spec3.json and spec3.yaml as well
  "project": "StripeAutogen",
  "output": "./output",
  "target": "fsharp",
  "synchronous": true,
  "asyncReturnType": "async",
  "resolveReferences": false,
  "emptyDefinitions": "ignore"
  //[ "overrideSchema" ],<JSON schema subset>,
  //[ "filterTags" ]
}

EDIT: I get exactly the same error when I use "schema": "thisfiledoesnotexist.yml", in other words, this appears to happen prior to opening the file, as with a non-existing file I receive the same error...

Version of Hawaii:

hawaii --version
0.59.0

Some OData responses need to be wrapped in a special OData type

Similar to #21, most OData responses come back with a field odata.context.

So we need a generic type

type OData<'a> =
    {
        [<JsonProperty("@odata.context")>]
        ODataContext: string
        value: 'a
    }

and instead of generating something like

[<RequireQualifiedAccess>]
type Extract =
    ///Success
    | OK of payload: SomeType
    ///error
    | DefaultResponse of payload: odataerror

the following is needed

[<RequireQualifiedAccess>]
type Extract =
    ///Success
    | OK of payload: OData<SomeType>
    ///error
    | DefaultResponse of payload: odataerror

in cases that return the odata context.

Object reference not set to an instance of an object. in create type abbreviation for array type

Hi Zaid,

i tried out Hawaii! Pretty cool stuff.

I found one bug in a special OpenAPI from cumulocity (https://cumulocity.com/api/dist/c8y-oas.json)

If you don't have a items inside the topLevelObject if the type is array then Hawaii will fail with a Object reference not set to an instance of an object error.

Hawaii will fail here:

let elementType = topLevelObject.Value.Items

I just wrapped a try with arround that and ignored that issue.

Could you have look into that? Maybe you have a better solution.

Thanks!

Idea - take advantage of existing OpenAPIParser

Hello friend, I am looking forward to what your next awesome thing will be. When I saw you're working with OpenAPI, I was thinking: maybe take a look at the existing F# parser lib I did some time ago: https://github.com/Dzoukr/OpenAPIParser

It has some limits but it can easily parse yaml (or json) into definition https://github.com/Dzoukr/OpenAPIParser/blob/master/src/OpenAPIParser/Version3/Specification.fs and that could be a good starting point for the generator you are writing.

Just an idea. Discard this message when read. ๐Ÿ˜„ All the best with the new Zaid's awesomeness! ๐Ÿ’ช

Possible type generation issue

Hi,

An observation from looking at some generated code (I haven't had chance to look at it in any detail, it's just something I spotted).

I have a schema (generated by NSwag from an ASP.Net Core service) which contains this snippet in the components section:

"ProblemDetails": {
  "type": "object",
  "additionalProperties": {
    "nullable": true
  },
  "properties": {
    "type": {
      "type": "string",
      "nullable": true
    },
    "title": {
      "type": "string",
      "nullable": true
    },
    "status": {
      "type": "integer",
      "format": "int32",
      "nullable": true
    },
    "detail": {
      "type": "string",
      "nullable": true
    },
    "instance": {
      "type": "string",
      "nullable": true
    },
    "extensions": {
      "type": "object",
      "nullable": true,
      "additionalProperties": {}
    }
  }
},
"ValidationProblemDetails": {
  "allOf": [
    {
      "$ref": "#/components/schemas/ProblemDetails"
    },
    {
      "type": "object",
      "additionalProperties": {
        "nullable": true
      },
      "properties": {
        "errors": {
          "type": "object",
          "nullable": true,
          "additionalProperties": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
    }
  ]
}

(ProblemDetails and ValidationProblemDetails are classes built into ASP for error reporting fwiw, where the latter is a subclass of the former).

The current Hawaii code generator generates this:

type ProblemDetails = Map<string, Newtonsoft.Json.Linq.JToken>
type ValidationProblemDetails =
    { ``type``: Option<string>
      title: Option<string>
      status: Option<int>
      detail: Option<string>
      instance: Option<string>
      extensions: Option<Map<string, string>>
      errors: Option<Map<string, list<string>>> }
    ///Creates an instance of ValidationProblemDetails with all optional fields initialized to None. The required fields are parameters of this function
    static member Create (): ValidationProblemDetails =
        { ``type`` = None
          title = None
          status = None
          detail = None
          instance = None
          extensions = None
          errors = None }

and that bit with the Map<string, Newtonsoft.Json.Linq.JToken> looks wrong to me - it looks odd how ValidationProblemDetails contains all of the properties from ProblemDetails, but ProblemDetails is defined as a map?

Thanks.

Question: Why does my generated type have a member of type `string option` instead of `ActualType option`?

I generated with the schema: "https://selectapi.datascope.refinitiv.com/RestApi/v1/Authentication/$metadata".

One of the types that is generated is

type ActionImportRequestTokenPayload =
    { Credentials: Option<string> }

but it should be

type ActionImportRequestTokenPayload =
    { Credentials: Option<DataScopeSelectApiAuthenticationCredentials> }

Looking at the --from-odata-schema output, I see a few cases of anyOf, and I see from the readme section that anyOf is not supported.

Is the current absence of support the reason these types get represented as strings?

Response Header Parameter

Hello, I would like to implement a post api that returns a custom header param in the header.
I have seen that the sendAsync function returns the response text and status code.

let sendAsync (method: HttpMethod) (basePath: string) (path: string) (extraHeaders: Header list) (parts: RequestPart list) : Async<int * string> =
    async {
        let requestPath = applyPathParts path parts
        let requestPathWithQuery = applyQueryStringParameters requestPath parts
        let fullPath = combineBasePath basePath requestPathWithQuery
        let! response =
            Http.request fullPath
            |> Http.method method
            |> applyJsonRequestBody parts
            |> applyMultipartFormData parts
            |> applyUrlEncodedFormData parts
            |> Http.headers extraHeaders
            |> Http.withCredentials true
            |> Http.send

        let status = response.statusCode
        let content = response.responseText
        return status, content
    }

Is there currently a way to access the response header? Or does the library need to be extended?

Microsoft Graph OpenAPI schema produces invalid code (compile errors)

https://github.com/microsoftgraph/msgraph-metadata/tree/master/openapi/beta

This builds a project successfully, but that project fails to compile... after 15 minutes of turning my computer into a space heater:

3>Types.fs(21895,31): Error FS1219 : The union case named 'Tags' conflicts with the generated type 'Tags'
3>Types.fs(24405,31): Error FS1219 : The union case named 'Tags' conflicts with the generated type 'Tags'
3>Client.fs(12,6): Error FS0883 : Invalid namespace, module, type or union case name
3>Client.fs(89972,71): Error FS0001 : This expression was expected to have type
    'microsoftgraphchromeOSOnboardingStatus'    
but here has type
    'string'
3>Client.fs(89989,74): Error FS0001 : This expression was expected to have type
    'microsoftgraphchromeOSOnboardingStatus'    
but here has type
    'string'
3>Client.fs(109192,82): Error FS0001 : This expression was expected to have type
    'microsoftgraphglobalDeviceHealthScriptState'    
but here has type
    'string'
3>------- Finished building project: Integrations.MicrosoftGraph. Succeeded: False. Errors: 6. Warnings: 3194
Build completed in 00:15:24.498

Schema was acquired using wget, jq and yj to get it into a suitable form:

wget https://github.com/microsoftgraph/msgraph-metadata/raw/master/openapi/beta/default.yaml
cat default.yaml | yj -yj | jq > default.json

The config is something like:

{
  "schema": "./default.json",
  "output": "../../gen/Integrations.MicrosoftGraph",
  "project": "Integrations.MicrosoftGraph",
  "synchronous": true,
  "emptyDefinitions": "free-form",
  "asyncReturnType": "task",
  "target": "fsharp"
}

Thank you for making this project as well!

Integer OverflowException when deserializing an int64 whose value is too large to fit in an int32

Hi,

I have an OpenApi schema whose return types include some Integer type fields that have a format of int64.
The Hawaii code generator generates client properties of type int64, but if I try to call the server function and it returns a value which is too large to fit in an int32, then I get this:

image

I had a go at debugging it and I think the error comes from

https://github.com/Zaid-Ajaj/Fable.Remoting/blob/ea1a681100bcfcb02accf81e1fcad6899e7d1a32/Fable.Remoting.Json/FableConverter.fs#L452

where it tries to convert the JSON value into an int, before upcasting to a (u)int64 (and then fails if the value is too large for an int32)?

It actually appears to work ok if the target client property is an Option<int64> rather than a plain int64 (different code paths in the json converter I think?).

Fwiw, I actually fell over this while having a look at making Snowflaqe understand custom 'Long' types (ref Zaid-Ajaj/Snowflaqe#8 (comment)) and then tried in Hawaii as well (given that int64 is part of the main spec here, where it's an extension in GraphQl)

Multiple client request parameters as a record

I need to work with an API which has multiple request header parameters. The client methods were generated as

    member this.PostMyMethod
        (
            prm1: string,
            prm2: string, 
...
            ?prmN: someType,
        )

I have nearly 10 parameters. While this could be done I really do not like passing 10 parameters if I could pass a single record object. I did not find in any options if I could specify wrapping all of them into a record type. Can I do this?

Why I need this?
If I have 5 optional parameters, if I create a record with all of them manually then to perform the correct calls with optional parameters and their permutations it is quite a lot of calls need to be written. I can not pass an Option as a value as I need to pass T.

Paths level Parameters are ignored

I have to parse a json which has parameters on Path level.

I did a quick fix by extending operationParameters and calling it in createOpenApiClient like this:

let parameters = operationParameters operationInfo pathInfo.Parameters visitedTypes config

pathInfo.Parameters are concated with the operation.Parameters. I haven't checked yet if that does the overriding like specified in the OpenAPI documentation.

Does that sound like a valid PullRequest, or am I missing something? :)

System.NullReferenceException

This is an exciting project so I thought I would try it out with my client's API but, alas, it fails:

{
  "schema": "https://api.podiotsuite.com/docs/swagger/swagger.json",
  "project": "podiotsuite",
  "output": "podiotsuite",
  "synchronous": true
}

Urls containing [ and ] generates code that does not compile

If the URL contains [ ] the types name will contain it as well and generate compilation errors.

According to https://fsharp.org/specs/language-spec/4.0/FSharpSpec-4.0-latest.pdf in section "3.4 Identifiers and Keywords"

_Note that when an identifier is used for the name of a types, union type case, module, or namespace, the following characters are not allowed even inside double-backtick marks:
โ€˜.', '+', '$', '&', '[', ']', '/', '\', '*', '"', '`'

Maybee exchange thoose chars to something or remove them when generating the Client.fs?

feature request: create methods with optional parameters

Instead of

type MyType =
    {
        A:int option
        B:int option
    }
    static member Create () : MyType =
        { A = None; B = None}

How about

static member Create(?a: int, ?b:int) = {A = a; B = b}

That way I can create with only the values that I need, very easily.

System.NullReferenceException on Homebridge schema

Just tried Hawaii 0.55.0 on Homebridge OpenApi schema with hawaii.config

{
    "schema": "./Homebridge.OpenAPI.json",
    "output": "./Homebridge.API",
    "project": "Homebridge.API",
    "target": "fable",
    "synchronous": false,
    "asyncReturnType": "async"
}

and it failed with NRE


Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Program.readParamType@1927(CodegenConfig config, OpenApiSchema schema) in C:\projects\Hawaii\src\Program.fs:line 1928
   at Program.readParamType@1927(CodegenConfig config, OpenApiSchema schema) in C:\projects\Hawaii\src\Program.fs:line 1953
   at Program.operationParameters(OpenApiOperation operation, List`1 visitedTypes, CodegenConfig config) in C:\projects\Hawaii\src\Program.fs:line 2043
   at Program.createOpenApiClient(OpenApiDocument openApiDocument, List`1 visitedTypes, CodegenConfig config) in C:\projects\Hawaii\src\Program.fs:line 2277
   at Program.runConfig(String filePath) in C:\projects\Hawaii\src\Program.fs:line 3140
   at Program.main(String[] argv) in C:\projects\Hawaii\src\Program.fs:line 3268

Would be nice to provide more details about the issue if schema is in unexpected format.

Improper management of connection errors

this is one of the get calls coded by the generator

    member this.GetGetMdp() =
        async {
            let requestParts = []
            let! (status, content) = OpenApiHttp.getAsync url "/get-mdp" headers requestParts
            return GetGetMdp.OK(Serializer.deserialize content)
        }

If the call fails (for example because of network disconnection), getAsync doesn't throw. The returned status is 0 and the content is empty "". Then the code attempts to deserialize "" and obviously fails with a serialization exception. The point is, there is no way for the user to distinguish genuine json problems from network connection problem, as both result in a json parsing exception.

Is there a way to differentiate between them?

uuid response type as string and not converted to Guid

Return type with payload of type System.Guid is created, but the client tries to create the return type with the string content directly instead of creating a System.Guid first.

Example:

  • specification.json
{   "paths": { "/auth/repo/": { "post": {
   "operationId": "PostAuthRepo",
   "responses": { "201": { "content": { "application/json" : { "schema": {
      "type": "string",
      "format": "uuid" }}}}}}}}}
  • Types.fs
[<RequireQualifiedAccess>]
type PostAuthRepo =
    | Created of payload: System.Guid
    | Unauthorized
    | Forbidden
  • Client.fs
member this.PostAuthRepo(repository: string, body: Authorization) =
        async {
            let requestParts =
                [ RequestPart.path ("repository", repository)
                  RequestPart.jsonContent body ]

            let! (status, content) =
                OpenApiHttp.postAsync url "/authorization/repositories/{repository}" headers requestParts

            if status = 201 then
                return PostAuthorizationRepository.Created content
            else if status = 401 then
                return PostAuthorizationRepository.Unauthorized
            else
                return PostAuthorizationRepository.Forbidden
        }

return PostAuthorizationRepository.Created content gives an error and should be return PostAuthorizationRepository.Created (System.Guid content)

Generation of nullable required property

Hi Zaid,

I have an API definition which contains a model with an required property with the datatype nullable string.
Hawaii generates the type as string. I expect it to be a string option.

required = parameter.Required || not nullable

As I understand it correct, this line is responsible if a type is optional or not.
Maybe the following replacement is an option?

parameter.Required && not nullable

Maybe there is an reason for only checking the required property?

Thanks in advance.

Fails on Application Insights Open API spec

Tested this URI out.

Hawaii correctly generates a project with some types; however, the project does not build with several errors e.g.

The type 'WorkbookProperties' is not defined

WorkbookProperties does not appear to be a oneOf case from what I can see.

Support for TimeSpan types in responses?

Hi,

A question\request about possible support for TimeSpan types in generated F# clients.

Say that you have an ASP.Net Core API that returns an object with properties of type System.TimeSpan (e.g. https://github.com/Numpsy/OpenApiExample/blob/timespans/Common/SpecTestController.cs), using NewtonSoft.Json as a serializer, and you use NSwag to generate a schema for the server.

That will generate an OpenApi schema such as this

"schemas": {
  "TimeSpanHolder": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "timeSpan": {
        "type": "string",
        "format": "time-span"
      }
    }
  }
}

So the TimeSpan is represented as a string with the custom format time-span.
If you point NSwags own C# client generator at that, it will recognize the custom format and generate

[System.CodeDom.Compiler.GeneratedCode("NJsonSchema", "10.5.2.0 (Newtonsoft.Json v11.0.0.0)")]
public partial class TimeSpanHolder 
{
    [Newtonsoft.Json.JsonProperty("timeSpan", Required = Newtonsoft.Json.Required.DisallowNull, NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore)]
    public System.TimeSpan TimeSpan { get; set; }
}

And I was wondering if you'd have any thoughts about doing the same in the Hawaii F# generator?

If you do the same thing but generate the server schema with Swashbuckle then you get this schema

"schemas": {
    "TimeSpanHolder": {
      "type": "object",
      "properties": {
        "timeSpan": {
          "type": "string",
          "format": "date-span"
        }
      },
      "additionalProperties": false
    }
  }

Which is similar except the custom format is date-span rather than time-span, because who needs consistency :-( (NSwags client generator seems to generate plain string properties for this one)

Make possible to pass a cancellation token

I propose to add an option to the configuration to generate code with CancellationToken for asynchronous calls.
For example, asyncReturnType could have 4 possible values instead of 2:
["asyncReturnType"]: <"async" | "task" | "async-ct" | "task-ct">

Nullable types in Dictionary values don't seem to be treated as plain Nullable values

type

type Trade() = 
    [<Required>] member val EntryUtc : DateTimeOffset = _default with get, set
    [<Required>] member val EntryGrid : sbyte = _default with get, set
    member val ExitUtc : Nullable<DateTimeOffset> = _default with get, set
    member val ExitGrid : Nullable<sbyte> = _default with get, set
    interface IYatpGraphItem with member this.Utc = this.EntryUtc

schema json

      "Trade": {
        "required": [
          "EntryGrid",
          "EntryUtc"
        ],
        "type": "object",
        "properties": {
          "EntryUtc": {
            "type": "string",
            "format": "date-time"
          },
          "EntryGrid": {
            "type": "integer",
            "format": "int32"
          },
          "ExitUtc": {
            "type": "string",
            "format": "date-time",
            "nullable": true
          },
          "ExitGrid": {
            "type": "integer",
            "format": "int32",
            "nullable": true
          }
        },

Hawaii.NET

type Trade =
    { EntryUtc: System.DateTimeOffset
      EntryGrid: int
      ExitUtc: Option<System.DateTimeOffset>
      ExitGrid: Option<int> }
    ///Creates an instance of Trade with all optional fields initialized to None. The required fields are parameters of this function
    static member Create (entryUtc: System.DateTimeOffset, entryGrid: int): Trade =
        { EntryUtc = entryUtc
          EntryGrid = entryGrid
          ExitUtc = None
          ExitGrid = None }

Nullables become Options, as expected

INSTEAD

type

type Mdp() = 
    [<Required>] member val Utc : DateTimeOffset = _default with get, set
    [<Required>] member val Volume : int = _default with get, set
    [<Required>] member val Open : float32 = _default with get, set
    [<Required>] member val High : float32 = _default with get, set
    [<Required>] member val Low : float32 = _default with get, set
    [<Required>] member val Close : float32 = _default with get, set
    [<Required>] member val Wave : Dictionary<Episode, Nullable<float32>> = _default with get, set
    [<Required>] member val Indicators : Dictionary<IndicatorId, Nullable<float32>> = _default with get, set
    [<Required>] member val Cards : Dictionary<CardId, CardValue> = _default with get, set
    interface IYatpGraphItem with member this.Utc = this.Utc

schema json

     "Mdp": {
        "required": [
          "Cards",
          "Close",
          "High",
          "Indicators",
          "Low",
          "Open",
          "Utc",
          "Volume",
          "Wave"
        ],
        "type": "object",
        "properties": {
          "Utc": {
            "type": "string",
            "format": "date-time"
          },
          "Volume": {
            "type": "integer",
            "format": "int32"
          },
          "Open": {
            "type": "number",
            "format": "float"
          },
          "High": {
            "type": "number",
            "format": "float"
          },
          "Low": {
            "type": "number",
            "format": "float"
          },
          "Close": {
            "type": "number",
            "format": "float"
          },
          "Wave": {
            "type": "object",
            "additionalProperties": {
              "type": "number",
              "format": "float",
              "nullable": true
            }
          },
          "Indicators": {
            "type": "object",
            "additionalProperties": {
              "type": "number",
              "format": "float",
              "nullable": true
            }
          },
          "Cards": {
            "type": "object",
            "additionalProperties": {
              "type": "string",
              "nullable": true
            }
          }
        },

Note nullable=true are present

Hawaii.NET

type Mdp =
    { Utc: System.DateTimeOffset
      Volume: int
      Open: float32
      High: float32
      Low: float32
      Close: float32
      Wave: Map<string, float32>
      Indicators: Map<string, float32>
      Cards: Map<string, string> }
    ///Creates an instance of Mdp with all optional fields initialized to None. The required fields are parameters of this function
    static member Create (utc: System.DateTimeOffset,
                          volume: int,
                          ``open``: float32,
                          high: float32,
                          low: float32,
                          close: float32,
                          wave: Map<string, float32>,
                          indicators: Map<string, float32>,
                          cards: Map<string, string>): Mdp =
        { Utc = utc
          Volume = volume
          Open = ``open``
          High = high
          Low = low
          Close = close
          Wave = wave
          Indicators = indicators
          Cards = cards }

Dictionaries becomes Maps, but the value type doesn't respect nullable=true (should be Option<_>)

As a result:

 ---> Newtonsoft.Json.JsonSerializationException: Error converting value {null} to type 'System.Single'. Path 'M', line 1, position 171949.
 ---> System.InvalidCastException: Null object cannot be converted to a value type.

Note also that if I change manually the generated type from Map<string, t> to Map<string, t option> then it works correctly

Header Parameter

I have a range paramter that belongs in the Header.
Hawaii recognizes that and creates:

async {
    let requestParts =
        [ ...
          if interval.IsSome then
              RequestPart.query ("interval", interval.Value)
          if range.IsSome then
              RequestPart.header ("range", range.Value)
          if name.IsSome then
              RequestPart.query ("name", name.Value)
          ... ]

    let! (status, content) =
        OpenApiHttp.getAsync url "/repositories/{repository}/timeseries" headers requestParts

but in sendAsync only the parameter extraHeaders is used for Headers.
So ranges work if I define them as part headers in the class instance of my client, but not as part of my json attributes.

Suggested solution:
Filter Header definitions from parts in sendAsync and merge them with extraHeaders.

Remove (or have an option to remove) null/None values in the serialized content

I am working with an API that complains about an incorrect date format on an optional field because the body contains something like "datefield": null

If I comment out fields that I don't use so that they don't appear in the json, then it works fine.
Would be good to be able to suppress the serialization of null/None values.

Edit: the date format error is not the only error. This happens even with optional integer fields for example.

OpenApi schema field descriptions containing new lines

Hi,

A small possible issue I noticed by accident when testing something.

Say that I have an OpenApi schema that contains something like

"status": {
    "type": "string",
    "description": "pet status\r\nin the store",
    "enum": [
        "available",
        "pending",
        "sold"
    ]
}

Then i get this for the generated type

image

Which looks like the doc comment has been split over two lines, when I guess it should be collapsed in to one, or have /// on both lines? (fwiw, I have a couple of schemas that are generated from ASP.Net Core services which do that if a comment on the .NET type is multi-line).

Thanks.

How to get the "@odata.type" entries in the serialized content?

As is, the generated code creates a body that looks something like this

{
	"ExtractionRequest": {
		"ContentFieldNames": [
			"RIC",
			"Round Lot Size",
			"ISIN",
			"Currency Code"
		],
		"IdentifierList": {
			"InstrumentIdentifiers": [
				{
					"Identifier": "IBM.N",
					"IdentifierType": "Ric"
				}
			]
		},
		"Condition": {
			"FixedIncomeRatingSources": "None"
		}
	}
}

But the API I am using will only work if I POST it something like this

{
	"ExtractionRequest": {
		"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TermsAndConditionsExtractionRequest",
		"ContentFieldNames": [
			"RIC",
			"Round Lot Size",
			"ISIN",
			"Currency Code"
		],
		"IdentifierList": {
			"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
			"InstrumentIdentifiers": [
				{
					"Identifier": "IBM.N",
					"IdentifierType": "Ric"
				}
			]
		},
		"Condition": {
			"FixedIncomeRatingSources": "None"
		}
	}
}

For reference the API I am using is found at "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/$metadata"

Testing with APIs.guru

Here (https://apis.guru/browse-apis/) you can find about 3.6k different real-world schema definition that you can use for testing.

This https://api.apis.guru/v2/list.json json captains catalog of schema urls with version information.
You can read it, choose schemas that you like and run Hawaii on these schemas

Some code may be reused from here

APIs.guru was very useful when I wrote my swagger(v2) parser, helped my find infinity recursion on compilation of recursively-dependent types and helped me validated that compiler is able to compile generated types.

return values not included

Using this api https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.2-develop the generated types of the resolved json api are looking like this:

[<RequireQualifiedAccess>]
type GetApi02EntityById =
    ///A geojson representation of the specified entity
    | OK
    ///Something went wrong. Please consult the error message.
    | NotFound

and are missing the actual content. The client method is consequently also not returning it:

    member this.GetApi02EntityById(id: float, ?show: string, ?download: bool, ?export: string, ?format: string) =
        let requestParts =
            [ RequestPart.path ("id_", id)
              if show.IsSome then
                  RequestPart.query ("show", show.Value)
              if download.IsSome then
                  RequestPart.query ("download", download.Value)
              if export.IsSome then
                  RequestPart.query ("export", export.Value)
              if format.IsSome then
                  RequestPart.query ("format", format.Value) ]

        let (status, content) =
            OpenApiHttp.get httpClient "/api/0.2/entity/{id_}" requestParts

        if status = HttpStatusCode.OK then
            GetApi02EntityById.OK
        else
            GetApi02EntityById.NotFound

Add capability to also generate Openapi spec file from F# project?

https://khalidabuhakmeh.com/generate-aspnet-core-openapi-spec-at-build-time

it would be nice if hawaii also allowed for openapi spec generation in yaml or json format,

as currently swashbuckle doesnt work for functional aspnetcore frameworks (Giraffe/Falco/Saturn etc..).

So could be really useful but I understand is something completely different and maybe this project is not the right place?

this is not supported/usable
https://github.com/rflechner/SwaggerForFsharp
pimbrouwers/Falco#49 (comment)
domaindrivendev/Swashbuckle.AspNetCore#2300

Thank you so much for hawaii

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.