netlify / open-api Goto Github PK

I see there's Implicit Grant implemented.
But I'm having trouble implementing a server-side Authorization Code Grant.

is it available but not documented? or unavailable at this time?

Situation: Enterprise customer Global Citizen Year is unable to successfully deploy their site from git. While there is another problem also blocking, this one is 1/2 of the problem. They have unicode filenames and their deploys error like this:

failed during stage 'deploying site': Failed to execute deploy: Upload cancelled: updates/on-walking-sharing-space-and-our-mother%c2%89s-garden/index.html

(from https://app.netlify.com/sites/wizardly-wing-6da16b/deploys/5bac50cd73f2cf0ea39d04ca)

you can see the problem more clearly in humio:

Failed to upload file updates/on-walking-sharing-space-and-our-mother%c2%89s-garden/index.html" build_id=5bac50cd73f2cf0ea39d04cb context=production deploy_id=5bac50cd73f2cf0ea39d04ca error="[PUT /deploys/{deploy_id}/files/{path}][422] uploadDeployFile default u0026{Code:422 Message:Missing entry in deploy for /updates/on-walking-sharing-space-and-our-motheru0089s-garden/index.html}

(from https://cloud.humio.com/netlify-production/search?widgetType=list-view&query=5bac50cd73f2cf0ea39d04ca%20on-walking-sharing-space&live=false&start=1537945200000&end=1538031600000&fullscreen=false)

@bcomnes tracked this down to this repo and behavior:

When you make the API call to create a deploy for files that contain unicode characters in filenames, filenames in the initial manifest need to be unescaped (== actual names), but when uploaded (PUT to the API) must be escaped since the filename is in a URL

The Identity service adds endpoints for user management, like

https://api.netlify.com/api/v1/sites/{SITE_ID}/identity/{IDENTITY_INSTANCE_ID}/users/invite

We also need to add identity_instance_id to the payload of the site endpoint.

Ref: https://github.com/netlify/product/issues/166

The https://open-api.netlify.com/#/default spec is missing the https://api.netlify.com/oauth/token endpoint

This is used to fetch OAuth creds like so:

curl -d 'grant_type=client_credentials&client_id=<YOUR_CLIENT_ID>&client_secret=<YOUR_CLIENT_SECRET>' https://api.netlify.com/oauth/token

From calling operations.DeleteSite:

panic: interface conversion: interface {} is *operations.DeleteSiteDefault, not *operations.DeleteSiteOK

goroutine 205 [running]:
github.com/netlify/open-api/go/plumbing/operations.(*Client).DeleteSite(0xc42032d7e0, 0xc42039b110, 0x1db43e0, 0xc4203d75d0, 0x30, 0x1894dc0, 0x1930501)
	/Users/mitchellh/code/go/src/github.com/netlify/open-api/go/plumbing/operations/operations_client.go:400 +0x56e

Build notifications from Netlify haven't been arriving to this repo in a while.

I suspect that many of the endpoints are incomplete - have had attributes added or they should be PUT instead of PATCH. A review to set a good baseline would be useful

I can see form the UI that these both exist but the API doesn't support this yet. Both would be great for Terraform. Thanks! 🙇

Hi! Is there a way to get the cloud functions usage (requests & hours) for a particular site via the API? If not, would this be something which could be added? Thanks 😁

Not mentioned at all, but used in creating dns records

On creating a Netlify site with no settings site (params are empty), I'm periodically getting this:

netlify_site.test: &{0 } (*models.Error) is not supported by the TextConsumer, can be resolved by supporting TextUnmarshaler interface

Sorry, it is in Go => string format.

This is in a loop of creating/deleting a site with no params. Seems to be about 1 in 10 chance.

We should be running some linters on the go client.

Title says it all. How does that work? This seems to be the best option for gdpr compliance.

We have a bunch of new team entrypoints that are not here, we need to add them to the swagger.yml file.

There are 25 open issues on this repo (he says as he adds to the noise). @futuregerald should review them to see what makes sense and what's been fixed and what's a WONTFIX

Right now, there is only a "default" section. We should divide the API by logical sections:

• sites
• deploys
• dns
• hooks
• forms
• snippets
  ...

Currently when customers do something "interesting" with functions (there's a large class of things that count as interesting; see related issue: https://github.com/netlify/bitballoon/issues/1628), we give a very non-useful error message while failing the deploy:

10:37:50 AM: Failing build: Failed to deploy site
10:37:50 AM: failed during stage 'deploying site': Failed to execute deploy: Upload cancelled: submission-created

This comes from https://github.com/netlify/buildbot/blob/d1d46200977e9388ad296974ec8dacb61a1c47bc/bot/stage_deploy.go#L50

@calavera asserts that it should be improved here:

https://github.com/netlify/open-api/blob/master/go/porcelain/deploy.go#L389

if sharedErr.err != nil {
  return fmt.Errorf("Upload cancelled: %s -- %v", f.Name, sharedErr.err)

And maybe once that messaging is improved, buildbot can be changed to surface the reason rather than just that there was an inscrutable error.

Please forgive me if I'm incorrect, but it appears we are missing the /user endpoint for listing user information.

I'm in the process of creating a .NET client for the API and am using the Open API specification as a guide along with the API docs. This is probably the first of several issues I'll open as I find discrepancies between the Open API specification, the API docs, and what I see on the wire. I'm happy to tackle some of these as PRs as well, but want to get them in as issues first.

Both the API docs and the wire data contain a payload for a processing_settings property in the site and siteSetup objects;

"processing_settings": {
    "css": {
      "bundle": true,
      "minify": true
    },
    "js": {
      "bundle": true,
      "minify": true
    },
    "images": {
      "optimize": true
    },
    "html": {
      "pretty_urls": true
    },
    "skip": true
  }

This structure appears to be totally missing from the Open API specification.

Hello,

For CreateSite, the API allows us to populate the repo field with something like mitchellh/foo. However, upon reading that site, only the repo_url is populated and repo is empty.

For Terraform, this is making it difficult to verify the remote settings are setup correctly. If this is working as intended I can try to parse this information out but the best thing would be to avoid that if possible.

The createSite operation in the Open API specification uses the siteSetup definition for the body but that object appears to contain many more properties than are relevant for the operation. The API docs only mention name, custom_domain, password, force_ssl, processing_settings (see #47), and repo. Wondering if a new object definition should be created and referenced for the createSite operation?

I am wondering if it would be possible to have a Delete deploy.

Currently we can delete a site

Is it possible to have a DELETE /deploys/{deploy_id} ? or is that not possible?

A file is opened to compute the SHA, but keeps the file handle open. If the deploy has a large number of files, this can result in a too many open files error.

The wire data contains a plan_data property for the site definition:

"plan_data": {
    "title": "Netlify Team Free",
    "asset_acceleration": true,
    "form_processing": true,
    "cdn_propagation": "partial",
    "build_gc_exchange": "buildbot-gc",
    "build_node_pool": "buildbot-ssd",
    "domain_aliases": true,
    "secure_site": false,
    "prerendering": true,
    "proxying": true,
    "ssl": "custom",
    "rate_cents": 0,
    "yearly_rate_cents": 0,
    "cdn_network": "free_cdn_network",
    "branch_deploy": true,
    "managed_dns": true,
    "geo_ip": true,
    "split_testing": true,
    "id": "nf_team_dev"
  }

This appears to be missing from the Open API specification.

When I create a site with a linked GitLab repository using the createSite operation, the site doesn't respond to incoming events (push or merge request) from the webhook (https://api.netlify.com/hooks/gitlab).

Here's the API call I'm using to create the site (which contains a netlify.toml file):

const Netlify = require('netlify')
const client = new Netlify(process.env.NETLIFY_TOKEN)

;(async () => {
  const { id: deployKeyId } = await client.createDeployKey()
  const site = await client.createSite({
    body: {
      repo: {
        provider: 'gitlab',
        deploy_key_id: deployKeyId,
        repo_path: 'group/repo-name',
        repo_branch: 'master',
      }
    }
  })
})()

If I click "Edit settings", then "Link to a different repository", the site starts responding to events from the webhook. However, there doesn't seem to be any change in the site settings (at least as reported by the API).

Is the createSite API not capable of setting up a site with CI properly? If so, what setting am I missing?

I also noticed that even if the repository is public, and I set the public_repo property to true, Netlify still reports the repository as private (as indicated by the lock icon next to the repository URL on the Build Settings page and confirmed by the API).

Howdy! 👋

SiteBuildSettings is missing a lot of really important fields I wanted to point out:

• env
• provider
• repo_type
• repo_url
• repo_branch

I'm able to create a site connected to a VCS from the API but I can't reload that data and verify. This is particularly important in Terraform so it can reconcile if anything changed remotely.

EDIT: Added env as an important one since I can't modify env vars until this is available.

Now that go modules are out, we should start using go modules with the go lib.

Noticed there were issues for missing endpoints but didn't see one for /sites/{site_id}/traffic_splits, feel free to close if it's because it's still in beta!

Started to implement this TF resource but noticed this:

I believe that should be a BuildHook.

Thanks!

Form submissions can be deleted via the api, but this is not currently in open-api.

Reference: https://app.intercom.io/a/apps/q245f50x/inbox/inbox/conversation/15701865881

The API docs say:

Takes all the same parameters as when creating a site.

That is specified in the API docs as name, custom_domain, password, force_ssl, processing_settings (see #47), and repo. However, the Open API specification indicates the body should contain the much more expansive siteSetup object.

We should mark properties in the definitions as required when it's necessary. That way the generator will create validations for all the POST endpoints to verify those fields in the client side.

See the simple model example: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#simple-model

Hello,

I'm trying to understand how I can enable SSL with Let's Encrypt (which would correspond to POST /api/sites/{site_id}/ssl) with the go library. It looks like noting is defined in the library for this.
Is there something I didn't find?

Moreover, once it will be enabled, is there a way to disable it?
According to https://www.netlify.com/docs/api/#sites, there is no way to disable it...

I have to look into why the retryable requests fail. They should rewind the body and keep it open until the request succeeds or the attempts are reached.

res, err := b.apiClient.DeployFiles(absFilePath, absFuncPath, b.Branch, b.GitRef(), b.SysLog)

When the retry uploads fail with a 408 you end up with an error like:

deploying site': Failed to execute deploy: no consumer: "text/html; charset=UTF-8"

Not very friendly. We probably need to special case handle 408s since its not returning JSON messaging.

Would be great if the pages could be used as intended, to run queries against our live API. My understanding is that that has never worked, but could? I may be mistaken.

cc @calavera

You used to be able to easily link to endpoints by clicking on them which would give you the anchor link in the url. This no longer works. Fix this to close this issue.

Sorry for all the issues, but I figure separate issues for each thing I find is easier to track and resolve than one big mega issue. If you'd prefer a mega issue then let me know.

Site Build Hooks are unsupported, specifically the API /sites/:id/build_hooks.

I'm using Netlify client from porcelain package for site operations (create, deploy..).
I want to implement site delete action (using site id), but I'm getting error type *porcelain.Netlify has no field or method DeleteSite.
I tried adding DeleteSite method to site.go in porcelain package (using GetSite from Netlify client as a pattern) just to see would it work, and it did, site got deleted.

Are you planning on adding this method to Netlify client?

I did manage to make it work by calling Operations.DeleteSite on Netlify client, but I also had to import plumbing operations package in order to get NewDeleteSiteParams. Site gets deleted, but this just doesn't look like a preferred way of doing it.

import(
    netlify "github.com/netlify/open-api/go/porcelain"
    netlifyOperations "github.com/netlify/open-api/go/plumbing/operations"
)

func deleteNetlifySite(siteID string) {
    
    ...

    client := netlify.New(transport, strfmt.Default)
    resp, err := client.Operations.DeleteSite(netlifyOperations.NewDeleteSiteParams().WithSiteID(siteID), authInfo)
}

Fix the watch command to rebuild the site automatically.

Each API operation should have summary, a description and at least a tag. That way the documentation will be more complete. Descriptions support GitHub Favored Markdown too:

https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#operation-object

I'm working on a Terraform provider and noticed there is no read/delete endpoints for deploy keys.

The primary sketchy thing about this for me is that Terraform can't verify that a deploy key is still valid.

From a Netlify PoV, we'll likely be running nightly acceptance tests at some point that could create dozens if not a hundred or more deploy keys. Is it okay that these grow unbounded?

There are more dns_zones endpoints in the api that are undocumented. Add them to open-api

The release process for this is very manual currently. Automate this so everyone can run it consistently:

https://github.com/netlify/open-api/blob/master/CONTRIBUTING.md#making-a-new-release

Netlify seems awesome! I saw this:

Currently, we're generating client code for Go, but we're planning on releasing libraries in any language that can generate code from the spec.

.. but creating an issue anyway, so that I can track and be notified when other languages are supported.

I am particularly looking for Scala / Java support.

Thanks!

Hi there,

am I correct in assuming that there is currently no way of GETting the list of files for an older deploy? Something that would look like /deploys/{deploy_id}/files. I just found uploadDeployFile but it is a PUT.

Basically, I need to check the hash assigned to a particular file name in several old deploys recursively (they are too many to do it by hand).

Thanks!

netlify/netlifyctl#109 has the details - and is blocked on this fix.