openservicebrokerapi / servicebroker Goto Github PK
View Code? Open in Web Editor NEWOpen Service Broker API Specification
Home Page: https://openservicebrokerapi.org/
License: Apache License 2.0
Open Service Broker API Specification
Home Page: https://openservicebrokerapi.org/
License: Apache License 2.0
Bindings return several pieces of information as part of the credential response. One key set of data is endpoint information necessary for connecting to the service. There is currently no way of differentiating this information, but it is critical to be able to access from an automated system for establishing network connectivity in various ways as part of the binding consumption process.
This problem is exacerbated by the fact that many different brokers expose endpoint information under different field names, e.g., ip
, host
, hostname
, address
.
We need the ability to understand endpoint information about a binding, including both address/hostname and port, whether it is by annotation in a schema (requires #116), or by making these concrete fields.
Proposal is introduced as part of the proposal for #116: https://docs.google.com/document/d/1JbsJgqgNtqthcfYwK_KbS6C8sjElrZNgoLhu40dUPAs/edit?usp=sharing
Currently bindings don't have names, I think they should. That would make it easier to keep track of bindings, would allow multiple bindings between serviceA and serviceB, say a single database, but two connections to it? @bmelville you had another use case for it too, I don't recall...
Context
cloudfoundry/cloud_controller_ng#567
https://www.pivotaltracker.com/n/projects/966314/stories/115985379
As a service author I want to be able to flag bindable
at the plan level, so I can have granular control
The bindable attribute should work by selecting the most specific value. For example:
Service level: bindable=true
Plan level: bindable=false
Result: the plan is not bindable
Service level: bindable=false
Plan level: bindable=true
Result: the plan is bindable
Bindable at the plan level should be optional and default to the service level attributes value.
{
"services": [{
"name": "fake-service",
"id": "acb56d7c-XXXX-XXXX-XXXX-feb140a59a66",
"description": "fake service",
"tags": ["no-sql", "relational"],
"requires": ["route_forwarding"],
"max_db_per_node": 5,
"bindable": true,
"metadata": {
"provider": {
"name": "The name"
},
"listing": {
"imageUrl": "http://example.com/cat.gif",
"blurb": "Add a blurb here",
"longDescription": "A long time ago, in a galaxy far far away..."
},
"displayName": "The Fake Broker"
},
"dashboard_client": {
"id": "398e2f8e-XXXX-XXXX-XXXX-19a71ecbcf64",
"secret": "277cabb0-XXXX-XXXX-XXXX-7822c0a90e5d",
"redirect_uri": "http://localhost:1234"
},
"plan_updateable": true,
"plans": [{
"name": "fake-plan",
"id": "d3031751-XXXX-XXXX-XXXX-a42377d3320e",
"description": "Shared fake Server, 5tb persistent disk, 40 max concurrent connections",
"max_storage_tb": 5,
"bindable": true, #optional
"metadata": {
"cost": 0,
"bullets": [{
"content": "Shared fake server"
}, {
"content": "5 TB storage"
}, {
"content": "40 concurrent connections"
}]
}
}]
}
}]
}]
}
The term plan is mentioned several times in the spec but it isn't clearly defined. It would be good to have it explained in the spec.
Right now the spec is written like its a subset of the CF docs, which is it :-) but we need to make it more stand-alone and so a re-org is probably needed. While doing that we should make it more normative by adding the appropriate RFC2119 keywords to make it clear what's required/optional for people wanting to implement the spec.
Binding credentials are a generic map of data, and as such are difficult to reason about or program against in a meaningful way. Similar to issue #59, we would like to support the ability to declare the schema of credential outputs from a binding.
Proposal: https://docs.google.com/document/d/1JbsJgqgNtqthcfYwK_KbS6C8sjElrZNgoLhu40dUPAs/edit?usp=sharing
Hi Guys,
Really like seeing this effort taking place. I was wondering if there is or should be a reference implementation and a test suite that can be run against brokers implementing the spec?
Following on from #126.
It was discussed that pagination is required since the catalog may respond with 10k+ entries. This would also mean that the paged response could include hundreds of pages or more. The scenario that I wanted to use to demonstrate a need for search is like so.
A user on the client platform is building an application and wants to look for NoSQL services that they can leverage. If the catalog API responds with all the entries and hundreds of pages, it becomes very difficult for someone to search through the catalog itself to locate what they may want to use.
I was suggesting the need to support search on the catalog API itself. The broker would be responsible for filtering entries accordingly and then return a paged response.
To seed discussion it might be a good start to allow searches on tags
and possibly description
in the service
object. I also think it might be helpful to allow someone to filter services that have or do not have free plans.
These could be query parameters on the catalog api either by allowing for the properties themselves ?tags=nosql
or allowing a fixed parameter like ?query=tags:nosql
. The second might allow more operators like AND, OR etc.
the spec implies it because planID is a required field in instance.create() but we should be explicit about it.
In order for bindings to be of any value, they must be able to model relationships between two service instances. The current CF model for bindings is a CF app to a service instance, so this must be extended.
The initial naive proposal for PoC purposes is:
Concretely, this is:
Open questions:
https://docs.cloudfoundry.org/services/api.html#binding
This is for the registry->SB/instance side of #26
We need some guidance on how to get the proper auth token.
This is all currently optional, so I'm not sure how important it is to nail it down as part of a spec. Discussion did seem rather concerned with uses that allowed charging. I was trying to write down a swagger definition and having trouble with this part of the definition of a plan object.
cost is defined on a plan.
api.md defines it as 'cost'
catalog-metadata.md defines it as 'costs'
the structure of the 'cost' object is defined in terms of usd, which is obviously somewhat centered on a specific currency. As well as using that currency as a key in an object, which is not good.
It is unclear about how multiple costs combine together if they exist.
The spec is out of date. Rebase against latest changes in https://github.com/cloudfoundry/docs-services, including changes for v2.10 of the spec.
looks like there are missing docs
These are all links on https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md that 404 on github for me (basically any link that isn't self referencing spec.md):
https://github.com/openservicebrokerapi/servicebroker/blob/devguide/services/index.html
https://github.com/openservicebrokerapi/servicebroker/blob/master/v2-api-changelog.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/app-log-streaming.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/route-services.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/volume-services.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/catalog-metadata.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/dashboard-sso.md
https://github.com/openservicebrokerapi/servicebroker/blob/master/managing-service-brokers.md#register-broker
https://github.com/openservicebrokerapi/servicebroker/blob/master/api.md#broker-errors
https://github.com/openservicebrokerapi/servicebroker/blob/master/examples.md
https://github.com/openservicebrokerapi/servicebroker/blob/devguide/services/managing-services.html#update_service
this is a 404 on cloudfoundry.org
http://docs.cloudfoundry.org/running/managing-cf/quota-plans.md
I only made it about 20% though spec.md before creating this issue, but it looks like the supporting docs might not have been migrated.
https://docs.cloudfoundry.org/services/api.html
Basically, the service registry->service broker calls related to #24
https://docs.cloudfoundry.org/services/api.html#provisioning
While reading the specification it was not clear to me what happens to bindings during a deprovision call. Are the bindings orphaned? or are they actively deleted by the broker? or should deprovision be disallowed if there are active bindings?
https://github.com/openservicebrokerapi/servicebroker/blob/master/_spec.md#asynchronous-operations
https://github.com/openservicebrokerapi/servicebroker/blob/master/_spec.md#request-2
Following the discussion in #107 this issue aims to create a proposal for "Service Broker Actions".
As a service broker author, I want to be able to optionally offer operations beyond "provision, update, bind, unbind, delete" so that I can provide new features to users of my service via the service broker API
Service clients can:
Service authors can:
#107
https://docs.google.com/document/d/1Y5vwWjvaUIwHI76XU63cAS8xEOJvN69-cNoCQRqLPqU/edit#heading=h.kzfeyua6md59
We have come across this use-case several times in various places. Given the output of a Bind() operation, we would like to be able to understand which fields within the credential are secret and must be treated so, and which are non-secret and may be treated less securely.
For example, if the output of a binding is:
credential:
username: admin
password: mypassword
hostname: localhost
port: 8080
username
and password
may be treated as a secret in UI or in credential handling by the controller, while hostname
and port
may be treated as general configuration data.
Proposal is introduced as part of the proposal for #116: https://docs.google.com/document/d/1JbsJgqgNtqthcfYwK_KbS6C8sjElrZNgoLhu40dUPAs/edit?usp=sharing
I understand the spec offers service plan updates (https://github.com/openservicebrokerapi/servicebroker/blob/master/_spec.md#updating_service_instance) but I'm concerned with a scenario where the service instance itself has changed.
Consider a case where a service provider offers a service which is "single tenant provisioned postgresql" and at the time a user provisions an instance of that service, they get their own postgresql v9.4 server stood up.
Later, the service provider decides that they are going to offer postgresql v9.5 instead. So users who provision from that day forward get a v9.5 server. Is there a workflow for legacy users to request their existing v9.4 instances get updated? Is the assumption that the user will have to deprovision their instance and provision a new one? Or that the service provider is responsible for handling updating provisioned instances outside the scope of a service catalog/service broker flow? Or should the v9.5 offering be registered as a new "plan"? (Is it an abuse of "plan" to use plan to handling the versioning of the catalog offering?)
As a service user I want to have confidence that my service instance is backed up and know that if something bad happens that I can restore from a backup.
As a service operator I want to offer service users a self-service mechanism to perform backup and restores of the service instance, so I don't have to manage this process.
There should be a way to update a binding object. Once a binding is created, one can only delete it. But there are some use cases where being able to support update on Binding would be nice to have. For example, a user of a service could apply quota adjustment that would be on per binding. The model proposed for Service Instance actions could be one way to achieve this.
Other use cases might include credential rotation as well as route modifications.
Proposal from @gberche-orange:
https://docs.google.com/document/d/1DoAbJa_YiGIJbOZ_zPzakh7sc4TB9Tmadq41cfSX0dw/edit?usp=sharing. Discussion of that proposal: http://cf-dev.70369.x6.nabble.com/cf-dev-service-broker-user-delegation-beyond-service-dashboard-tt2658.html
Proposal from @fraenkel: https://docs.google.com/document/d/1w4swx_5erf_NmPtfcCW2fn2jbEBfRHf2xLVhWpc1PLM/edit
Given that a standardized API will be implemented and consumed by more parties, possibly accessed over the open Internet, does it make sense to allow and/or standardize on other auth mechanisms (i.e. such as OAuth?). The Spec currently specifies Basic Auth as the only mechanism for authentication.
EDIT by @avade
Working document: https://docs.google.com/document/d/1XLPMLJbf_lRXIdMg1XUyKNZUd2--fDROGvdEm25tMjc/edit
Tracking issue
To prove our abstraction works beyond local storage.
Service broker will provide schema information for both service instances and bindings for each plan. The schema information is for both parameter inputs and output properties. Currently the proposal is that the schema format will be JSON schema v4.
Proposal: https://docs.google.com/document/d/1-IKI-PwPnhtbK0su1UzWA_UoyaCjL8S2d6ky3Xmuwvg/edit?usp=sharing
The broker API has since been versioned to 2.10. Rebase or submit an additional PR with 2.10 changes?
Why were the sections on the version header and authentication removed? Don't these deserve some explanation? They appear in example curls, but there is no other explanation. How should the broker author know what their responsibilities are?
With the exception of the table of contents, none of the links work.
The example response body for Provision contains "dashboard_url". The backslash should be removed. This may have been in an older version of github.com/cloudfoundry/docs-services but has since been fixed there.
The first paragraph of the intro for Updating a Service Instance is out of date. I recall rewriting that some time ago.
The whole section titled "Manage Application Requests with Route Services" can be removed. This link appears in the index of the current docs, but I've just sent a PR to remove it. This topic is for marketplace end-users (app developers), not for service broker authors. cloudfoundry/docs-services#145
Remove the whole section titled "Volume Services (Experimental/Obsolete)" and fix the link in TOB for "Volume Services (Experimental)"
You can remove the section "Using cf curl" under Access Control, as well as instructions to get the api version. I've just submitted a PR to docs-services to do the same. cloudfoundry/docs-services#147
courtesy of @duglin. not sure if these all require action, but I put them into a checklist
https://github.com/cloudfoundry/docs-services this is the current source of truth
It would be useful if there were a little more detail in the spec around API versioning. How is versioning defined here and for what intention? What behaviour is expected of a client and server (e.g. if their versions don't match)?
Examples:
This is for the client->service registry flow.
I'm referring to the following CF cf
calls:
service-brokers List service brokers
create-service-broker Create a service broker
update-service-broker Update a service broker
delete-service-broker Delete a service broker
rename-service-broker Rename a service broker
purge-service-offering Recursively remove a service and child objects from Cloud Foundry database without making requests to a service broker
The spec leaves open what service broker authors have to accept as a valid instance_id.
When we look at the following request:
PUT /v2/service_instances/:instance_id
On some places the specs explicitly uses a guid .
/v2/service_instances/:guid/...
In my opinion the spec could more precise when we reduce the instance_id to only guid values allowed and a specific representation (dashes, lowercase letters, ...)
That's also true for some other id fields where the spec currently says a "GUID is recommended".
From #95 (comment)
In v2.10 volume services are no longer experimental.
Note that there was a breaking change for volume services from v2.9 to 2.10
when the "experimental" flag was removed.As part of this WG I would love to discuss the use of "experimental" API
specs. Not sure that it worked well in this case.
Binding to an instance may encapsulate a long running workflow, similar to creating an instance. This might include provisioning new user accounts in a service, setting up firewalls, etc. It seems like the Bind API would benefit from the same last_operation model that Provision uses, and I'd like to get feedback from others who may have more context around this.
Latest proposal: https://github.com/mattmcneeney/servicebroker/blob/async-bindings/spec.md#binding
As a service broker author I can choose to ignore CF-isms in the Service Broker API specification and see that these are deprecated, but still available, just not as a top level attribute
Branch with spec changes: https://github.com/duglin/servicebroker/tree/CFisms
Diff of changes: master...duglin:CFisms
OLD: Google doc for the original proposal profile: https://docs.google.com/document/d/15X9LWzZw8EF5TSufqsdrBcdlALr-6rWdMo3FVCbjrfs/edit#heading=h.9gcbm1qmz9wc
cf
actions:
bind-service Bind a service instance to an app
unbind-service Unbind a service instance from an app
The spec contains some links that result in a 404 not found
Some of the links are part of a paragraph that looks like it is referencing itself or an incomplete paragraph.
E.g. #application-log-streaming
Application Log Streaming
...
For details, see Application Log Streaming.
We probably can't use the CF cli for all operations so we should just create a cli for just the user facing service related operations we're going to support.
We often try to use font icons (such as http://fontawesome.io/icons/ ) for images in UIs to reduce the need to load a large number of unique images. It would be good to have the option to define a CSS class for the image for a Service or Plan instead of requiring a unique URL for an image so that font icons can be used for these. UI's should be able to handle the existence of either an imageClass or an imageURL. This allows brokers that are knowledgeable about the frontends they are being deployed on to use known classes to minimize unique images that need to be hosted, but still allows brokers that are expected to be deployed across many environments to specify URLs for their images.
The service catalog can provide plans with costs. When a service is provided from a third party there is no specification or recommendation how to pass billing or account information to the service provider.
A possible solution could be to extend the plan metadata by required parameters for the service creation
Plan Metadata Field:
"metadata":{
...
"requires": [ "accountNumber" ],
...
}
When creating a service instance:
{
....
"parameters": {
"accountNumber": 123456789
}
}
Likely removed inadvertently in #131
I'm not sure if this has been discussed yet but I tough it is interesting to define what to do with the current metadata and community driven standards in OpenSB.
Currently Cloud Foundry SB API has a metadata field. As part of the docs, some community driven standards are included.
http://docs.cloudfoundry.org/services/catalog-metadata.html
This is positive because:
Issue #94 raises the point that this data may need a pass of refinement at the moment. But then that raises the bigger question about what to do with this spec in the migration to OpenSB.
Should it also be migrated and kept as an accessory document?
https://docs.cloudfoundry.org/services/api.html
this is on the call from the service registry to a service broker
Now that multiple pages have been squished into one, all the links must be updated.
The catalog API should support pagination in the cases where there are large
numbers of services supported by a broker. We see a potential for 10k-15k
catalog entries which could take a bit of time to render to the caller.
Effectively the initial catalog call should expect the Link header or the
metadata in the body indicating that the broker should be invoked with
pagination.
The API should take the following optional parameters:
parameter name | description |
---|---|
per_page | number of items to show per page |
page | page number requested, depends on page size. if > last page, then last page is returned |
$ curl -H "X-Broker-API-Version: 2.9" http://username:password@broker-url/v2/catalog?page=2&per_page=20
Response field | Type | Description |
---|---|---|
services* | array-of-service-objects | Schema of service objects defined below. |
pagination | metadata about pagination | Schema for pagination defined below. |
The pagination
block would be added to the catalog response at the same
level as services
. In the case of requesting the first page, the response
will contain a next and a last item.
{
"services": [{
...
}],
"pagination": {
"next": "https://broker/v2/catalog?page=2",
"last": "https://broker/v2/catalog?page=15333"
}
}
Alternatively we could use the Link header RFC 5988. In this case there would be NO changes to the response body just the headers.
Link: <https://broker/v2/catalog?page=2>; rel="next",
<https://broker/v2/catalog?page=15333>; rel="last"
Subsequent calls will yield 2 new options, first and prev. For example, if you request page 10000 the following response will be returned:
{
"services": [{
...
}],
"pagination": {
"next": "https://broker/v2/catalog?page=10001",
"last": "https://broker/v2/catalog?page=15333",
"first": "https://broker/v2/catalog?page=1",
"prev": "https://broker/v2/catalog?page=9999"
}
}
Alternatively we could use the Link header RFC 5988
Link: <https://broker/v2/catalog?page=10001>; rel="next",
<https://broker/v2/catalog?page=15333>; rel="last",
<https://broker/v2/catalog?page=1>; rel="first",
<https://broker/v2/catalog?page=9999>; rel="prev"
https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#-service-objects-
The requires
property lists (currently limited) set of syslog_drain
, route_forwarding
and volume_mount
. The permissions appear CF-specific. Is there plan to either extend this property to allow more general set of permissions, even implementation-specific ones, or is this field being removed as part of making the API more generic?
This github repo should be for development of the broker api specification. Example brokers could be in separate repos.
I would suggest that the only things that should be in this repo now are a README, license file, and the spec (w/images)
This is on the calls from a CLI to a service registry.
The cf
calls:
service Show service instance info
create-service Create a service instance
update-service Update a service instance
delete-service Delete a service instance
rename-service Rename a service instance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.