CockroachDB service broker for Pivotal Cloud Foundry
Introduction
The CRDB service broker allows binding applications to CockroachDB instances, providing "Level 2" integration (see pivotal docs).
A service broker is configured to expose one or more service plans. Each service plan is associated to a CockroachDB cluster or instance.
From service plans we can instantiate service instances. For each service instance, a database (namespace) is created on a CockroachDB instance.
Once we have a service set up, we can bind it to applications. Each binding
creates a unique user/password for that application and grants permissions on
the service database. The credentials are passed via a postgres:// URI;
applications which support Postgres may work without modifications (assuming
they don't use postgres-specific syntax that CockroachDB doesn't support).
Note that when we want different apps to be separated (different namespaces), they should use different service instances.
How to use
Prerequisites
- Install the CF CLI.
- Login to a CF instance.
- Set up one or more CockroachDB instances. These don't need to live on CF.
Note: for things to work with the sample
spring-musicapp, a recent version of CockroachDB which supports int4 must be used.
Installing the service broker
The service broker can either be pushed as an app, or deployed from a tile.
Using cf push
-
Edit
manifest.ymland edit thePRECONFIGURED_PLANSvariable. This variable is a list of service plans; each plan is configured to connect to a CockroachDB instance (this can be a single node, or a load balancer on top of a cluster). -
Push the service broker "app":
cf pushYou should see the service broker app and hostname under
cf apps. -
Create the service broker:
cf create-service-broker cockroachdb-service-broker user pass https://<hostname>The user and password are
SECURITY_USER_NAMEandSECURITY_USER_PASSWORDinmanifest.yml; the hostname is the one shown bycf apps.
The cf service-brokers command should show cockroachdb-service-broker.
Using the tile
The service broker is available as a CF tile. You can build the tile using the
Tile Generator by running
build.sh; then upload the product/*.pivotal file to CF (using the ops
manager). Then you can install the tile; there is a configuration form for
specifying details for the service plans.
Note that every build bumps the tile version. CF will barf if it sees two files with the same version that are different (even if the old one was uninstalled), so it's best to not circumvent the version change.
After the installation, the cf service-brokers command should show cockroachdb-service-broker.
Using the CockroachDB service
Make sure access to the service is enabled via cf service-access. To enable,
use cf enable-service-access cockroachdb. Then cf marketplace should list
the cockroachdb service with whatever plans the broker was pre-configured.
Create a service instance
Create a service for a specific plan:
cf create-service cockroachdb default crdb-service-1
If we log into the database, we can see that a new database has been created.
[email protected]:26257/> SHOW DATABASES;
+-------------------------------------+
| Database |
+-------------------------------------+
| cf_gbccnddiddnnhfdolliklaolojgmceif |
| crdb_internal |
| information_schema |
| lol |
| pg_catalog |
| system |
+-------------------------------------+
(6 rows)
Bind service instance to app
We can test things with a sample app called spring-music, which just exposes a
web UI to insert/remove music album information. The application supports a
postgres backend. To install it, see the spring-music-only-war repo.
Note: we can only use CockroachDB in insecure mode with this app (it tries to connect without TLS, regardless of the URI).
Once spring-music is pushed, we can bind it to our new service:
cf bind-service spring-music crdb-service-1
This operation created a new user with access to the database associated with
crdb-service-1:
[email protected]:26257/> SHOW USERS;
+----------------------------------+
| username |
+----------------------------------+
| dkbafmddnfmgofgghiflhkcbnkdleknp |
| test |
+----------------------------------+
(2 rows)
[email protected]:26257/> SHOW GRANTS ON DATABASE cf_gbccnddiddnnhfdolliklaolojgmceif;
+-------------------------------------+----------------------------------+------------+
| Database | User | Privileges |
+-------------------------------------+----------------------------------+------------+
| cf_gbccnddiddnnhfdolliklaolojgmceif | dkbafmddnfmgofgghiflhkcbnkdleknp | ALL |
| cf_gbccnddiddnnhfdolliklaolojgmceif | root | ALL |
+-------------------------------------+----------------------------------+------------+
We can take a look at the environment for spring-music, and notice it includes a URI to our database:
cf env spring-music
...
"uri": "postgres://dkbafmddnfmgofgghiflhkcbnkdleknp:[email protected]:26257/cf_gbccnddiddnnhfdolliklaolojgmceif?sslmode=disable",
...
Restart the app (cf restart spring-music) and it should now be using our database:
[email protected]:26257/> SHOW TABLES FROM cf_gbccnddiddnnhfdolliklaolojgmceif;
+-------+
| Table |
+-------+
| album |
+-------+
(1 row)
[email protected]:26257/> SELECT * FROM cf_gbccnddiddnnhfdolliklaolojgmceif.album;
+--------------------------------------+---------+---------------------------+-------+-------------+----------------------------+------------+
| id | albumid | artist | genre | releaseyear | title | trackcount |
+--------------------------------------+---------+---------------------------+-------+-------------+----------------------------+------------+
| 24c8fb84-9908-488a-aeb8-ab3ccae6df83 | NULL | Michael Jackson | Pop | 1982 | Thriller | 0 |
| 27ce02b2-f6d6-4679-bc61-4c059d015088 | NULL | Jimi Hendrix Experience | Rock | 1967 | Are You Experienced? | 0 |
| 292de30a-a1e6-4585-92af-f09d699ccb20 | NULL | Fleetwood Mac | Rock | 1977 | Rumours | 0 |
| 2e43897e-ff0c-4c77-975c-c8f3da6ae8fc | NULL | The Beatles | Rock | 1969 | Abbey Road | 0 |
| 31f90083-9dac-405b-af04-e8186210294b | NULL | The Rolling Stones | Rock | 1969 | Let it Bleed | 0 |
| 357512b3-480b-482c-8dd5-73db3158f548 | NULL | Led Zeppelin | Rock | 1969 | Led Zeppelin | 0 |
| 41fd0674-44fe-47d8-b4ae-6bc80f8431cb | NULL | Stevie Ray Vaughan | Blues | 1983 | Texas Flood | 0 |
| 4526d75e-aa48-4dd9-ac92-a6e8683818b2 | NULL | The Eagles | Rock | 1976 | Hotel California | 0 |
| 52e72d98-b535-435f-b688-1d7fe58985c6 | NULL | The Clash | Rock | 1980 | London Calling | 0 |
| 5a94e4e8-d656-40b3-8699-183baacdbdfb | NULL | Robert Johnson | Blues | 1961 | King of the Delta Blues | 0 |
| 609b5b41-4759-4ae3-b699-0c8356a9a409 | NULL | The Rolling Stones | Rock | 1972 | Exile on Main Street | 0 |
| 71b5ba16-4509-4966-b428-ebdb8a0706fb | NULL | The Ramones | Rock | 1976 | The Ramones | 0 |
| 7c02d294-81b3-4fae-bb65-accd0a9de511 | NULL | Boston | Rock | 1978 | Don't Look Back | 0 |
| 7dfb002c-f8ee-477a-bf5d-eb40b1689107 | NULL | The Fabulous Thunderbirds | Blues | 1979 | Rock With Me | 0 |
| 7e7d5ecb-419c-43cf-b004-cc50c2068ff7 | NULL | Queen | Rock | 1975 | A Night At The Opera | 0 |
| 875ca773-7b5f-4a56-bae5-cf762749c7c0 | NULL | Albert King | Blues | 1967 | Born Under A Bad Sign | 0 |
| 91e7751d-705b-4f1a-8d08-7910cfd294bd | NULL | Nirvana | Rock | 1991 | Nevermind | 0 |
| 93732bce-994c-45e4-b431-a36d7e92773d | NULL | Bruce Springsteen | Rock | 1975 | Born to Run | 0 |
| 98f80b7c-e28b-4aec-bfdc-f447e687d6c1 | NULL | The Beach Boys | Rock | 1966 | Pet Sounds | 0 |
| 9d6c043a-90c1-4764-aa6e-88ec42275617 | NULL | Marvin Gaye | Rock | 1971 | What's Going On | 0 |
| 9f394353-d0d6-413f-a392-3a9a8b23e625 | NULL | U2 | Rock | 1991 | Achtung Baby | 0 |
| a7aa43fb-c712-4204-8705-90b7ec27c426 | NULL | Police | Rock | 1983 | Synchronicity | 0 |
| a7ab9a9c-fca6-4f86-8023-d8ae6b19ae08 | NULL | The Beatles | Rock | 1965 | Rubber Soul | 0 |
| b8388fef-3df1-48b2-a9fe-2ceca6cfa5b0 | NULL | U2 | Rock | 1987 | The Joshua Tree | 0 |
| c72cdab7-d333-4502-83ac-1a95923d2dad | NULL | Stevie Ray Vaughan | Blues | 1984 | Couldn't Stand The Weather | 0 |
| d5fac975-ff76-42e9-aee4-57ed23032101 | NULL | Elvis Presley | Rock | 1976 | Sun Sessions | 0 |
| d7621a16-1fca-4878-935f-bc12ea20e72a | NULL | BB King | Blues | 1956 | Singin' The Blues | 0 |
| f03ca1a7-9cf3-4294-b447-6b237d9df058 | NULL | Led Zeppelin | Rock | 1971 | IV | 0 |
| f0a93b83-9029-4102-a208-b808a0628bc3 | NULL | Muddy Waters | Blues | 1964 | Folk Singer | 0 |
+--------------------------------------+---------+---------------------------+-------+-------------+----------------------------+------------+
(29 rows)
Kubernetes (experimental)
Kubernetes Service Catalog introduces the Open Service Broker API to Kubernetes. This means that Service Brokers written for Cloud Foundry can be easily ported to Kubernetes.
Containerize
The following commands will build a Docker container of the CockroachDB Service Broker and push it to your Docker Registry (make sure authentication is configured properly):
export CRDBSB_IMAGE=yourusername/crdbsb
export CRDBSB_TAG=latest
make docker-push
Deploy
Deployment has been successfully tested on minikube 0.28.2 with the following extra configuration:
% minikube start --container-runtime docker \
--memory 8192 \
--cpus 2 \
--extra-config=apiserver.Authorization.Mode=RBAC
You will also need to install the Service Catalog on your Kubernetes cluster.
Once your cluster is ready, make sure your kubectl context is configured correctly and issue the following commands to deploy the Service Broker:
export CRDBSB_IMAGE=yourusername/crdbsb
export CRDBSB_TAG=latest
export CRDB_HOST=cockroachdb-public.default.svc.cluster.local
export CRDB_PORT=26257
export CRDB_TOKEN=cockroachdb-cockroachdb-token-rs2kk
make deploy
You can configure your CockroachDB cluster endpoint using the CRDB_HOST
and CRDB_PORT environment variables.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent