Code Monkey home page Code Monkey logo

overview-broker's Introduction

Overview Broker

Job Status
Unit Unit status
Conformance Conformance status
Dockerhub Dockerhub status

A simple service broker conforming to the Open Service Broker API specification that hosts a dashboard showing information on service instances and bindings created by any platform the broker is registered with.

Other fun features this broker provides include:

  • Edit the broker catalog without redeploys to speed up testing
  • History of recent requests and responses
  • Ability to enable different error modes to test platform integrations
  • Change the response mode on the fly (sync only/async only/async where possible)
  • A range of configuration parameter schemas for provision service instance, update service instance and create service binding
  • Asynchronous service instance provisions, updates and deletes
  • Asynchronous service binding creates and deletes
  • Fetching service instances and bindings
  • Generic extensions for fetching the Health and Info for a service instance

What is the Open Service Broker API?

Open Service Broker API

The Open Service Broker API project allows developers, ISVs, and SaaS vendors a single, simple, and elegant way to deliver services to applications running within cloud native platforms such as Cloud Foundry, OpenShift, and Kubernetes. The project includes individuals from Fujitsu, Google, IBM, Pivotal, RedHat and SAP.

Quick start

Dockerhub

The latest version of overview-broker can always be found on Dockerhub. You can pull and run the latest image with:

docker pull ismteam/overview-broker
docker run ismteam/overview-broker

Build it

git clone [email protected]:cloudfoundry/overview-broker.git
cd overview-broker
npm install

# Start overview-broker
npm start

# Or to run the tests
npm test

Configuration

  • To set the BasicAuth credentials, set the BROKER_USERNAME and BROKER_PASSWORD environmental variables. Otherwise the defaults of admin and password will be used.
  • To expose a route service, set the ROUTE_URL environmental variable to a url. It must have https scheme.
  • To expose a syslog drain service, set the SYSLOG_DRAIN_URL environmental variable to a url.
  • To expose a volume mount service, set the EXPOSE_VOLUME_MOUNT_SERVICE environmental variable to true.
  • To generate many plans with a range of configuration parameter schemas, set the ENABLE_EXAMPLE_SCHEMAS environmental variable to true.
  • By default, all asynchronous operations take 1 second to complete. To override this, set the ASYNCHRONOUS_DELAY_IN_SECONDS environmental variable to the number of seconds all operations should take.
  • To specify how long platforms should wait before timing out an asynchronous operation, set the MAXIMUM_POLLING_DURATION_IN_SECONDS environmental variable.
  • To specify how long Platforms should wait in between polling the /last_operation endpoint for service instances or bindings, set the POLLING_INTERVAL_IN_SECONDS environmental variable to the number of seconds a platform should wait before trying again.
  • To change the name of the service(s) exposed by the service broker, set the SERVICE_NAME environmental variable.
  • To change the description of the service(s) exposed by the service broker, set the SERVICE_DESCRIPTION environmental variable.
  • To set the response mode of the service broker (note that this can also be changed via the broker dashboard), set the RESPONSE_MODE environmental variable to one of the available modes.
  • To set the error mode of the service broker (note that this can also be changed via the broker dashboard), set the ERROR_MODE environmental variable to one of the available modes.

Platforms

Cloud Foundry

1. Deploying the broker
  • First you will need to deploy the broker as an application:
    cf push overview-broker -i 1 -m 256M -k 256M --random-route -b https://github.com/cloudfoundry/nodejs-buildpack
  • You can also use an application manifest to deploy the broker as an application:
    wget https://raw.githubusercontent.com/cloudfoundry/overview-broker/master/examples/cloudfoundry/manifest.yaml
    cf push
  • The overview broker dashboard should now be accessible:
    open "https://$(cf app the-best-broker | awk '/routes:/{ print $2 }')/dashboard"
2. Registering the broker
  • To register the broker to a space (does not require admin credentials), run:
    cf create-service-broker --space-scoped overview-broker admin password <url-of-deployed-broker>
    The basic auth credentials "admin" and "password" can be specified if needed (see Configuration).
  • The services and plans provided by this broker should now be available in the marketplace:
    cf marketplace
3. Creating a service instance
  • Now for the exciting part... it's time to create a new service instance:
    cf create-service overview-service small my-instance
    You can give your service a specific name in the dashboard by providing the name configuration parameter:
    cf create-service overview-service small my-instance -c '{ "name": "My Service Instance" }'
  • If you now head back to the dashboard, you should see your new service instance!
4. Creating a service binding
  • To bind the service instance to your application, you will need to first push an application to Cloud Foundry with cf push. You can then create a new binding with:
    cf bind-service <app-name> my-instance

Kubernetes

1. Deploying the broker
  • Deploy the broker and a load balancer that will be used to access it:
    wget https://raw.githubusercontent.com/cloudfoundry/overview-broker/master/examples/kubernetes/overview-broker-app.yaml
    wget https://raw.githubusercontent.com/cloudfoundry/overview-broker/master/examples/kubernetes/overview-broker-service.yaml
    kubectl create -f overview-broker-app.yaml
    kubectl create -f overview-broker-service.yaml
    You can check this has succeeded by running kubectl get deployments and kubectl get services.
  • Once the load balancer is up and running, he overview broker dashboard should be accessible:
    open "http://$(kubectl get service overview-broker-service -o json | jq -r .status.loadBalancer.ingress[0].ip)/dashboard"
2. Registering the broker
  • To register the broker, you first need to install the Service Catalog. The instructions to do this can be found here. If service catalog fails to install due to permissions, you might want to look at this guide.
  • You should now be able to register the service broker you deployed earlier by creating a clusterservicebrokers custom resource:
    BROKER_URL="http://$(kubectl get service overview-broker-service -o json | jq -r .status.loadBalancer.ingress[0].ip)"
    cat <<EOF | kubectl create -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: overview-broker-secret
      namespace: default
    type: Opaque
    stringData:
      username: admin
      password: password
    EOF
    cat <<EOF | kubectl create -f -
    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ClusterServiceBroker
    metadata:
      name: overview-broker
      namespace: default
    spec:
      url: ${BROKER_URL}
      authInfo:
        basic:
          secretRef:
            name: overview-broker-secret
            namespace: default
    EOF
    Note that if you changed the default basic auth credentials (see Configuration), then you will need to change the Secret defined above.
  • The services and plans provided by this broker should now be available:
    kubectl get clusterserviceclasses
    kubectl get clusterserviceplans
3. Creating a service instance
  • Now for the exciting part... it's time to create a new service instance:
    cat <<EOF | kubectl create -f -
    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ServiceInstance
    metadata:
      name: my-instance
      namespace: default
    spec:
      clusterServiceClassExternalName: overview-service
      clusterServicePlanExternalName: small
    EOF
  • If you now head back to the dashboard, you should see your new service instance!
4. Creating a service binding
  • Creating a service binding with service-catalog will result in a new Secret being created which represents the information returned from the service broker for the binding. To create a new service binding, you can run:
    cat <<EOF | kubectl create -f -
    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ServiceBinding
    metadata:
      name: my-instance-binding
      namespace: default
    spec:
      instanceRef:
        name: my-instance
      secretName: my-instance-secret
    EOF
    To see the contents of the service binding, you can get the associated secret with:
    kubectl get secret my-instance-secret -o yaml
    Note that the data shown will be base64 encoded.

overview-broker's People

Contributors

blgm avatar christopherclark avatar crswty avatar dependabot-preview[bot] avatar dependabot[bot] avatar felisiam avatar georgi-lozev avatar jenspinney avatar mattmcneeney avatar samze avatar teddyking avatar waterlink avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

overview-broker's Issues

Maintenance info version must be a Semantic Version

The recent changes to add maintenance info support are resulting in a Semantic Version error:

$ cf version
cf version 6.47.2+d526c2cb3.2019-11-05

$ cf target
api endpoint:   https://api.run.nulldriver.com
api version:    2.139.0
user:           admin
org:            cfclitest Org 8kl4b3BOMW
space:          cfclitest Space 8kl4b3BOMW

$ cf create-service-broker 'cfclitest Space Scoped Broker 8kl4b3BOMW' admin password https://cfclitest-space-scoped-broker-8kl4b3bomw.cfapps.nulldriver.com --space-scoped
  Creating service broker cfclitest Space Scoped Broker 8kl4b3BOMW in org cfclitest Org 8kl4b3BOMW / space cfclitest Space 8kl4b3BOMW as admin...
  Service broker catalog is invalid: 
  Service overview-service
    Plan small
      Maintenance info version must be a Semantic Version, but has value "v1.0"
    Plan large
      Maintenance info version must be a Semantic Version, but has value "v1.0"
  
  FAILED

If I use the previous version of overview-broker before that commit, everything works fine.

Add a REST API to fetch recent requests and their responses

expected behavior

As an osb tooling author (such as osb-reverse-proxy)

  • in order to use overview broker recent requests and responses programmatically
  • I need a rest API

observed behavior

I could only find html output which is hard to parse

The html-xml-utils package is failing to parse overview output:

https://stackoverflow.com/questions/51913432/scraping-information-within-html-tags-in-unix-with-curl-and-cut/51913624#51913624

$ cat overview.html | hxremove title
End tag </script> doesn't match start tag <br>

Getting 502s when hitting the /dashboard

Hi @mattmcneeney!

We've recently noticed that the overview broker consistently returns 502s when we try to hit the dashboard endpoint. Our logs look like this:

  2018-12-14T10:55:50.07-0800 [CELL/0] OUT Starting health monitoring of container
   2018-12-14T10:55:50.57-0800 [APP/PROC/WEB/0] OUT > [email protected] start /home/vcap/app
   2018-12-14T10:55:50.57-0800 [APP/PROC/WEB/0] OUT > node index.js
   2018-12-14T10:55:50.98-0800 [APP/PROC/WEB/0] OUT Service broker created. (1 service exposed)
   2018-12-14T10:55:50.99-0800 [APP/PROC/WEB/0] OUT Overview broker running on port 8080
   2018-12-14T10:55:53.70-0800 [CELL/0] OUT Container became healthy
   2018-12-14T10:55:58.51-0800 [RTR/0] OUT overview.indigo-sentry.lite.cli.fun - [2018-12-14T18:55:58.504+0000] "GET / HTTP/1.1" 303 0 68 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36" "24.16.214.23
4:57026" "10.244.0.139:61000" x_forwarded_for:"24.16.214.234" x_forwarded_proto:"http" vcap_request_id:"abe80456-519f-4f2c-4b6d-119448c7f98c" response_time:0.013484404 app_id:"af726421-6be2-44ac-a8f4-c695a4c4e921" app_index:"0" x_b3_traceid:"4fd3121009ce6edd" x_b3_spanid:"4fd31
21009ce6edd" x_b3_parentspanid:"-"
   2018-12-14T10:55:58.51-0800 [RTR/0] OUT
   2018-12-14T10:55:58.51-0800 [APP/PROC/WEB/0] OUT GET / 303 68 - 4.878 ms
   2018-12-14T10:55:58.77-0800 [RTR/0] OUT overview.indigo-sentry.lite.cli.fun - [2018-12-14T18:55:58.604+0000] "GET /dashboard HTTP/1.1" 502 0 67 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36" "24.
16.214.234:57026" "10.244.0.139:61000" x_forwarded_for:"24.16.214.234" x_forwarded_proto:"http" vcap_request_id:"4cd99d2f-1993-4054-6d76-47bcb1d3b5be" response_time:0.170166944 app_id:"af726421-6be2-44ac-a8f4-c695a4c4e921" app_index:"0" x_b3_traceid:"c4d710fdae49248f" x_b3_span
id:"c4d710fdae49248f" x_b3_parentspanid:"-"
   2018-12-14T10:55:58.77-0800 [RTR/0] OUT
   2018-12-14T10:55:58.79-0800 [APP/PROC/WEB/0] OUT Exit status 137
   2018-12-14T10:55:58.79-0800 [CELL/SSHD/0] OUT Exit status 0
   2018-12-14T10:55:59.11-0800 [CELL/0] OUT Cell c14f5c47-72ea-41e6-afa0-9dcfb7540768 stopping instance 0f6d9064-0553-4676-7b8e-acd4
   2018-12-14T10:55:59.11-0800 [API/0] OUT Process has crashed with type: "web"
   2018-12-14T10:55:59.12-0800 [CELL/0] OUT Cell c14f5c47-72ea-41e6-afa0-9dcfb7540768 destroying container for instance 0f6d9064-0553-4676-7b8e-acd4
   2018-12-14T10:55:59.13-0800 [API/0] OUT App instance exited with guid af726421-6be2-44ac-a8f4-c695a4c4e921 payload: {"instance"=>"0f6d9064-0553-4676-7b8e-acd4", "index"=>0, "cell_id"=>"c14f5c47-72ea-41e6-afa0-9dcfb7540768", "reason"=>"CRASHED", "exit_description"=>"APP/PROC/
WEB: Exited with status 137", "crash_count"=>2, "crash_timestamp"=>1544813759097010868, "version"=>"871d9c0d-5367-4aeb-b192-2b80e37eeeff"}
   2018-12-14T10:55:59.13-0800 [CELL/0] OUT Cell c14f5c47-72ea-41e6-afa0-9dcfb7540768 creating container for instance ce1ba35f-d7cd-416b-7d17-22f3
   2018-12-14T10:55:59.72-0800 [CELL/0] OUT Cell c14f5c47-72ea-41e6-afa0-9dcfb7540768 successfully created container for instance ce1ba35f-d7cd-416b-7d17-22f3
   2018-12-14T10:55:59.94-0800 [CELL/0] OUT Cell c14f5c47-72ea-41e6-afa0-9dcfb7540768 successfully destroyed container for instance 0f6d9064-0553-4676-7b8e-acd4
   2018-12-14T10:56:01.55-0800 [CELL/0] OUT Starting health monitoring of container
   2018-12-14T10:56:02.04-0800 [APP/PROC/WEB/0] OUT > [email protected] start /home/vcap/app
   2018-12-14T10:56:02.04-0800 [APP/PROC/WEB/0] OUT > node index.js
   2018-12-14T10:56:02.45-0800 [APP/PROC/WEB/0] OUT Service broker created. (1 service exposed)
   2018-12-14T10:56:02.46-0800 [APP/PROC/WEB/0] OUT Overview broker running on port 8080
   2018-12-14T10:56:05.24-0800 [CELL/0] OUT Container became healthy
   2018-12-14T10:56:07.61-0800 [RTR/0] OUT overview.indigo-sentry.lite.cli.fun - [2018-12-14T18:56:07.600+0000] "GET / HTTP/1.1" 303 0 68 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/71.0.3578.80 Chrome/71.0.3578.80 Safari/537.36"
"209.234.137.222:10621" "10.244.0.139:61000" x_forwarded_for:"209.234.137.222" x_forwarded_proto:"http" vcap_request_id:"c989dca6-d983-4a3e-541c-f2eda49b1423" response_time:0.012588346 app_id:"af726421-6be2-44ac-a8f4-c695a4c4e921" app_index:"0" x_b3_traceid:"aee5044a673ca4f3" x
_b3_spanid:"aee5044a673ca4f3" x_b3_parentspanid:"-"
   2018-12-14T10:56:07.61-0800 [RTR/0] OUT
   2018-12-14T10:56:07.61-0800 [APP/PROC/WEB/0] OUT GET / 303 68 - 4.378 ms
   2018-12-14T10:56:07.73-0800 [RTR/0] OUT overview.indigo-sentry.lite.cli.fun - [2018-12-14T18:56:07.641+0000] "GET /dashboard HTTP/1.1" 502 0 67 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/71.0.3578.80 Chrome/71.0.3578.80 Safari
/537.36" "209.234.137.222:10621" "10.244.0.139:61000" x_forwarded_for:"209.234.137.222" x_forwarded_proto:"http" vcap_request_id:"4c5a647f-accd-4d43-5089-ed1271a8b752" response_time:0.094408019 app_id:"af726421-6be2-44ac-a8f4-c695a4c4e921" app_index:"0" x_b3_traceid:"8c165e4239
e207eb" x_b3_spanid:"8c165e4239e207eb" x_b3_parentspanid:"-"
   2018-12-14T10:56:07.73-0800 [RTR/0] OUT
   2018-12-14T10:56:07.75-0800 [APP/PROC/WEB/0] OUT Exit status 137
   2018-12-14T10:56:07.75-0800 [CELL/SSHD/0] OUT Exit status 0

Wondering if you've run into this as well or have any ideas what could be causing this? From poking around a bit and adding some log lines, it appears that the crash is happening somewhere at this line: https://github.com/mattmcneeney/overview-broker/blob/044c88e1bb4bf112f2265abad740888c157c4e34/app.js#L56

Overview broker requires instance name when updating a service instance

OSBAPI says, only the service instance guid is required.

Overview broker rejects requests from cf to update a service instance.

โ†’ cf update-service my-db -p complex
Updating service instance my-db as admin...
FAILED
Server error, status code: 502, error code: 10001, message: The service broker rejected the request to http://overview-broker.grove-flier.sapi.cf-app.com/v2/service_instances/d2cf0174-a1cd-4a2f-b96c-be3e94df4f3a?accepts_incomplete=true. Status Code: 400 Bad Request, Body: instance requires property "name"

Consistent service offering/plans guid across restarts

Expected behavior

As an overview-broker user deployed in cloudfoundry

  • in order to support application restarts while still serving incoming OSB requests
  • I need that offering/plans guid to not change across application restarts

Observed behavior

Following e11bba2 the uuids are randomly generated at application restart.

Consequently, if the overview broker was registered in cloudfoundry the following errors will be observed

  • service instance provisionning requests fail with message Could not find service 895749e2-8543-4f9d-8134-3a71b4a388c4, plan 8fb7951a-f309-496a-89b2-7cbdec5a3dcc
  • if there are existing service instances, errors will occur while trying to deprovision them with message Could not find service 895749e2-8543-4f9d-8134-3a71b4a388c4, plan 8fb7951a-f309-496a-89b2-7cbdec5a3dcc
  • if the broker registration in cloudfoundry is requested, then it fails with the following message, because the services are already registered with a same name but distinct ids
Server error, status code: 502, error code: 270012, message: Service broker catalog is invalid: 

Service names must be unique within a broker. Services with names ["overview-service"] already exist

Workaround

Systematically perform a cf purge-service-offering -f overview-service -b overview-broker prior to execute a cf update-service-broker

Suggested fix

  • Use consistent hash (e.g. md5) from names (respectively service name and service plan name) as Guids
  • Add an opt-in (e.g. OSB_GUID_OFFSET) which adds an offset to the consistent hash, enabling registration of multiple overview broker instances in K8S svcat (CF support multi service registration out of the box)

Overview-Broker remote image?

Hi,

I was wondering if you intend on uploading a docker image of the overview-broker somewhere - maybe on DockerHub or Quay?

This would greatly simplify the spinning up of a overview-broker deployment without having to clone/fork your repository.

If you don't have this planned, is it okay for others to upload their own images in personal container repositories?

Node modules vendoring?

cf push logs out this warning:

         PRO TIP: It is recommended to vendor the application's Node.js dependencies
         Visit http://docs.cloudfoundry.org/buildpacks/node/index.html#vendoring

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.