hashicorp / vault-service-broker Goto Github PK

when using a vault namespace the service broker puts its information correctly in to the vault. When I try to bind to a service the policy fails to create. It looks like the policy write is not appending the namespace to the policy write. I was set up with a vault token created based on the service-broker sample policy. I added all of the namespace entry points to the policy to cover coming in through nested namespace. /devops/cicd as namespace
NOTE: the token and policy I created where in the namespace.

2019-02-25T10:48:58.05-0600 [APP/PROC/WEB/0] OUT [DEBUG] creating new policy cf-8cdb5f48-0315-44da-b912-00ab1677ffb9 2019-02-25T10:48:58.16-0600 [APP/PROC/WEB/0] OUT [ERR] failed to create policy cf-8cdb5f48-0315-44da-b912-00ab1677ffb9: Error making API req uest. URL: PUT https://hcvault-nonprod.dell.com/v1/sys/policies/acl/cf-8cdb5f48-0315-44da-b912-00ab1677ffb9 Code: 403. Errors: * 1 error occu rred: * permission denied 2019-02-25T10:48:58.16-0600 [RTR/1] OUT vault-broker-hilarious-swan.ausvdc02.pcf.dell.com - [2019-02-25T16:48:57.745+0000] "PUT /v2/service_ instances/8cdb5f48-0315-44da-b912-00ab1677ffb9/service_bindings/f505dfa2-b30f-41aa-be38-f3d0b2da55ce HTTP/1.1" 500 428 284 "-" "HTTPClient/1.0 (2.8.3, ruby 2.4.2 (2017-09-14))" "10.32.27.76:58798" "172.16.1.8:61030" x_forwarded_for:"10.175.172.11, 10.32.27.76" x_forwarded_proto:"https" vcap_request_id:"d61ddf9e-989e-4f4f-465f-1f6ac642b9e1" response_time:0.416970695 app_id:"24180243-0a86-4e82-a3ab-8626ed5800fc" app_index:"0" x _b3_traceid:"f9e7bfaa05814ef6" x_b3_spanid:"f9e7bfaa05814ef6" x_b3_parentspanid:"-"

Goal

Add and update (without overwriting) secret into Vault

Vault server

• Version 0.9.3
• Install with Vault Bosh release

Issue

I read that it is possible Vault CLI KV patch. Also, by default the secret engine KV with the path secret/. https://www.vaultproject.io/intro/getting-started/secrets-engines.html

The service broker create path under cf/ and it's not using the KV by default. I try to modify enable the secret engine with cf/ path. When I try to create a new Vault service, I have this error

error alert message,Service broker error: failed to create mounts 

### Questions
* How can I fix it?
* How do you add and update token? Do you use the Vault cli KV?

We're setting up a Vault cluster with 3 nodes using Consul for the backend storage in HA mode. VAULT_ADVERTISE_ADDR environment variable is the address to advertise to bound applications as the Vault's address. If this is the single IP address of one of Vault nodes and when this node is unavailable, how can the app know what are the IP addresses of the other Vault nodes for connect to in the failover?

Do we need to add a load balancer in front of the Vault nodes and use the IP/DNS of the load balancer in VAULT_ADVERTISE_ADDR environment variable? It looked like the Vault service broker assumes that bound applications should access the Vault HA cluster via a load balancer, but could you please confirm.

If we need to provide a list of IP addresses of all Vault nodes in HA cluster to a bound app, so the app can directly connect to first Vault node and fail over to the other node addresses in the list if first node is unavailable. Does Vault service broker support the direct node connection with Vault HA cluster? We can put multiple IP addresses in VAULT_ADVERTISE_ADDR, but I'm not sure if it's recommended. Should the Vault service broker introduce another environment variable (VAULT_ADVERTISE_ADDRS) to hold multiple IP addresses for HA cluster use case?

I'd like to run this broker inside of a docker environment.
Can you add this to your dockerhub?
Is the correct approach to it to hashicorp/docker-hub-images?

Thank you!

The handleRenew function does not update the metadata stored in the generic secret backend for the renewed token.

After the first renewal the meta data residing on the broker mount cf/broker/instanceID/bindingID will go stale and there after the renewals won't happen and the logs will complain

2017-06-12T12:07:35.76-0700 [APP/PROC/WEB/0] OUT [DEBUG] renewing token with accessor dda8587a-9983-09ae-cd19-a45f5bd7bd16
2017-06-12T12:07:35.76-0700 [APP/PROC/WEB/0] OUT [WARN] aborting renewing expired token with accessor dda8587a-9983-09ae-cd19-a45f5bd7bd16

The above token under the accessor dda8587a-9983-09ae-cd19-a45f5bd7bd16 is still valid, and renew worthy, but the meta-data stored in the vault has gone stale prompting the above warning.

#vault read cf/broker/53d02677-5b17-4044-ad2d-00de655ceba5/cce5cf96-a531-4efe-8ecb-5f403504ba3f


Key             	Value
---             	-----
refresh_interval	768h0m0s
json            	{"Organization":"9a9fb216-42ae-4bfb-865e-69b44e256a98","Space":"d0cae1cd-e4e2-44f4-9718-a1f6d54d605b","Binding":"cce5cf96-a531-4efe-8ecb-5f403504ba3f","ClientToken":"bd444b78-10da-92ad-5f75-ddfb107d5fdb","Accessor":"dda8587a-9983-09ae-cd19-a45f5bd7bd16","LeaseDuration":432000,"Renew":"2017-06-05T04:59:55.382848343Z","Expires":"2017-06-10T04:59:55.382848343Z"}

#vault auth
Token (will be hidden):
Successfully authenticated! You are now logged in.
token: bd444b78-10da-92ad-5f75-ddfb107d5fdb
token_duration: 15488
token_policies: [cf-53d02677-5b17-4044-ad2d-00de655ceba5 default]

The enterprise edition of Vault supports namespaces and the service broker should too. The destination namespace is added to the HTTP request in the form of the X-Vault-Namespace header, or by calling SetNamespace(namespace string) using github.com/hashicorp/vault/api.Client.

Hi, Currently the namespaces created dynamically for secret mounts looks very random and tightly coupled to the GUIDs generated on a CF/PCF ecosystem. Can this be made a bit name based instead of GUID for ease of management ?

 "VCAP_SERVICES": {
  "hashicorp-vault": [
   {
    "credentials": {
     "address": "https://blah.amazonaws.com:8200/",
     "auth": {
      "accessor": "25afc1eb-5ef7-1cbf-2aca-62468ac913af",
      "token": "d3a9123e-5c19-7562-432e-ebb763bae079"
     },
     "backends": {
      "generic": "cf/7abe1cfa-4a17-407a-8bf9-09f1544951c7/secret",
      "transit": "cf/7abe1cfa-4a17-407a-8bf9-09f1544951c7/transit"
     },
     "backends_shared": {
      "organization": "cf/9a9fb216-42ae-4bfb-865e-69b44e256a98/secret",
      "space": "cf/d0cae1cd-e4e2-44f4-9718-a1f6d54d605b/secret"
     }
    },
    "label": "hashicorp-vault",
    "name": "aws-vault",
    "plan": "shared",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     ""
    ],
    "volume_mounts": []
   }

 cf org my-org --guid
9a9fb216-42ae-4bfb-865e-69b44e256a98

cf space my-space --guid
d0cae1cd-e4e2-44f4-9718-a1f6d54d605b

It looks pretty weird since it is very random

Ideally speaking these would have been better placed if it were mounted based on names.

cf/my-org/secret
cf/my-space/secret
cf/my-app/secret (instead of the instance id)

Can you please let us know which one is most relevant ?
https://github.com/cloudfoundry-community/vault-broker

Some implementations of the open service broker api check to make sure the id field of plan objects match regex [a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])? Would it be possible to switch https://github.com/hashicorp/cf-vault-service-broker/blob/master/broker.go#L301 to ID: fmt.Sprintf("%s-%s", b.serviceID, VaultPlanName), in order to support this validation?

Hi Team,

While Deploying cloud-foundry-vault-service-broker as below
https://www.hashicorp.com/blog/cloud-foundry-vault-service-broker

Whenever we do cf start vault-broker

I have tried putting but no use. Please guide

cf set-env vault-broker VAULT_TOKEN "*************"
cf set-env vault-broker SECURITY_USER_NAME "vadmin"
cf set-env vault-broker SECURITY_USER_PASSWORD "*********"
cf set-env vault-broker TRUST_CERTS "10.119.69.204:8200"
cf set-env vault-broker VAULT_SKIP_VERIFY "true"

I just deployed the vault service broker v0.5.3 with VAULT_SKIP_VERIFY: true for the environment variable and got this error when connecting to a test Vault cluster configured with a self-signed certificate.

[ERR] failed to start broker: failed to create mounts: Get https://10.9.202.7/v1/sys/mounts: x509: certificate signed by unknown authority

Another user ran into the same problem as logged in this issue ( #45). Based on the comments, it looked like the VAULT_SKIP_VERIFY env variable is not currently supported in the Vault service broker. When looking the source code (https://github.com/hashicorp/vault-service-broker/blob/master/main.go#L40), the service broker first calls this function from Hashicorp vault API ("github.com/hashicorp/vault/api"):

vaultClientConfig := api.DefaultConfig()

This function returns a vault config containing an HTTP client with the needed transport TLS configurations properly picked up from the environment variables (VAULT_SKIP_VERIFY) and should be able to connect to a Vault cluster with a self-signed certificate. However when the next statement (https://github.com/hashicorp/vault-service-broker/blob/master/main.go#L41) is executed, a new HTTP client is created for Vault client config,

vaultClientConfig.HttpClient = cleanhttp.DefaultClient()

As this new HTTP client does not perform any further transport TLS configurations from the environment variables such as VAULT_SKIP_VERIFY, it would fail to make an SSL connection to Vault cluster with a self-signed certificate.

Possible resolutions for supporting the Vault cluster with self-signed certificate:

Option 1. If we can remove the following statement,
vaultClientConfig.HttpClient = cleanhttp.DefaultClient()

and just use the HTTP client (cleanhttp.DefaultPooledClient) created as part of api.DefaultConfig(), but there is a warning message on using the DefaultPooledClient:

Do not use this function for transient clients as it can leak file descriptors over time. Only use this for clients that will be re-used for the same host(s).

therefore cleanhttp.DefaultPooledClient might not be suitable for the service broker.

Option 2: If we want to continue to use the cleanhttp.DefaultClient, we would need to add the following code to configure TLS configurations for the service broker (https://github.com/hashicorp/vault-service-broker/blob/master/main.go).

func main() {
...
// Setup the vault client
vaultClientConfig := api.DefaultConfig()
vaultClientConfig.HttpClient = cleanhttp.DefaultClient()
if err := configureHttpClient(vaultClientConfig); err != nil {
logger.Fatal("[ERR] failed to configure the HTTP client", err)
}
...
}

func configureHttpClient(config *api.Config) error {
transport := config.HttpClient.Transport.(*http.Transport)
transport.TLSHandshakeTimeout = 10 * time.Second
transport.TLSClientConfig = &tls.Config{
MinVersion: tls.VersionTLS12,
}
if err := http2.ConfigureTransport(transport); err != nil {
return err
}
if err := config.ReadEnvironment(); err != nil {
return err
}
config.HttpClient.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
config.Backoff = retryablehttp.LinearJitterBackoff
config.MaxRetries = 2
return nil
}

could you please take a look to see if the service broker code can be changed to support the Vault cluster with self-signed certificates?

Is there any plans to work with Pivotal to wrap this service broker into a PCF TILE through Partner Tile Deployment? As my company has paid for Enterprise Vault and also has Pivotal Cloud Foundry. It would be nice this wrapped as a PCF Tile, so it eases my time as a platform operator manual go through the steps that Tile Deployment would do in automated fashion. And it makes it easier for my developers to use Vault Credentials.

Hi there! Im trying to deploy your vault service broker on Bluemix cloud foundry. I followed the steps shown on your github readme but, but could not get the vault-broker to start after pushing to cf.

after running cf start for the first time, I received the following error during the staging process:

Staging...
-------> Buildpack version 1.7.5
file:///tmp/buildpacks/6ac565cf5df70e40a10479c1f20d4b09/dependencies/https___pivotal-buildpacks.s3.amazonaws.com_concourse-binaries_godep_godep-v62-linux-x64.tgz
DEPENDENCY MISSING IN MANIFEST: go 1.7
It looks like you're trying to use go 1.7.
Unfortunately, that version of go is not supported by this build pack.
The versions of go supported in this buildpack are:
- 1.6.1
- 1.6
- 1.5.3
- 1.4.3
If you need further help, start by reading: http://github.com/cloudfoundry/go-buildpack/releases.
Failed to compile droplet
Exit status 223
Destroying container
Successfully destroyed container

FAILED
BuildpackCompileFailed

I then changed the manifest.yml so that the GOVERSION env variable was go1.6.1 instead of go1.7. I also changed the GOVERSION variable in the Makefile to go1.6.1.

However, running the broker like this resulted in another error:

Staging...
-------> Buildpack version 1.7.5
file:///tmp/buildpacks/6ac565cf5df70e40a10479c1f20d4b09/dependencies/https___pivotal-buildpacks.s3.amazonaws.com_concourse-binaries_godep_godep-v62-linux-x64.tgz
-----> Installing go1.6.1... done
 !     Error: Cloud Foundry does not support the govendor package manager
 !     We currently only support the 'Godep' package manager for go apps
 !     For support please file an issue: https://github.com/cloudfoundry/go-buildpack/issues
 !     
Failed to compile droplet
Exit status 223
Staging failed: Exited with status 223
Destroying container
Successfully destroyed container

FAILED
BuildpackCompileFailed

I found someone post a similar error here: cloudfoundry/go-buildpack#40. I followed a suggestion to push the app with an updated go build pack like so:

cf push my_app -b https://github.com/cloudfoundry/go-buildpack

This led to the following error here:

2017-02-28T14:39:16.14-0500 [API/4]      OUT Created app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d
2017-02-28T14:39:18.52-0500 [API/8]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"route"=>"f7ca19bf-1564-403f-98a3-33fbfaf87393"})
2017-02-28T14:39:49.23-0500 [API/9]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"environment_json"=>"PRIVATE DATA HIDDEN"})
2017-02-28T14:39:57.92-0500 [API/2]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"environment_json"=>"PRIVATE DATA HIDDEN"})
2017-02-28T14:40:56.72-0500 [API/1]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"environment_json"=>"PRIVATE DATA HIDDEN"})
2017-02-28T14:41:18.43-0500 [API/0]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"environment_json"=>"PRIVATE DATA HIDDEN"})
2017-02-28T14:41:26.52-0500 [API/5]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"environment_json"=>"PRIVATE DATA HIDDEN"})
2017-02-28T14:41:45.88-0500 [API/5]      OUT Updated app with guid 68055fce-cc25-47a7-a8fe-20430fdf723d ({"state"=>"STARTED"})
2017-02-28T14:41:46.46-0500 [STG/0]      OUT Creating container
2017-02-28T14:41:54.51-0500 [STG/0]      OUT Successfully created container
2017-02-28T14:41:54.51-0500 [STG/0]      OUT Downloading app package...
2017-02-28T14:41:54.73-0500 [STG/0]      OUT Downloaded app package (212.1K)
2017-02-28T14:41:54.73-0500 [STG/0]      OUT Staging...
2017-02-28T14:41:57.43-0500 [STG/0]      OUT -------> Buildpack version 1.7.19
2017-02-28T14:41:57.84-0500 [STG/0]      OUT https://buildpacks.cloudfoundry.org/dependencies/godep/godep-v79-linux-x64-9e37ce0f.tgz`
2017-02-28T14:41:58.49-0500 [STG/0]      OUT https://buildpacks.cloudfoundry.org/dependencies/glide/glide-v0.12.3-linux-x64-5b2e71ff.tgz
2017-02-28T14:42:29.31-0500 [STG/0]      ERR Failed to compile droplet
2017-02-28T14:42:29.32-0500 [STG/0]      OUT Exit status 223
2017-02-28T14:42:29.37-0500 [STG/0]      ERR Staging failed: Exited with status 223
2017-02-28T14:42:29.55-0500 [API/6]      ERR Failed to stage application: staging failed

Not sure where to go from here. Any help would be appreciated!

Hello I'm getting this 403 error on binding the app to the vault service:

$ cf bind-service my-app sbil-vault
Binding service sbil-vault to app my-app in org SvcsIT-AnEOrg / space devops as Rafael_Prado...
Service broker error: failed to create policy cf-eabb9d6c-2f3c-43ef-a76d-4e0e25eda128: Error making API request.
URL: PUT http://svcsltslab1.com/v1/sys/policies/acl/cf-eabb9d6c-2f3c-43ef-a76d-4e0e25eda128

Code: 403. Errors:
* 1 error occurred:
* permission denied
FAILED

I handled it by adding this to the cf-broker.hcl:

path "sys/policies/acl/cf-*" {
  capabilities = ["create", "update", "delete"]
}

This is not on the README.md doc, I was going to generate a PR for it but as I'm very new to vault and I don't know if it's correct to add this policy I believe that I should first ask about it here :)

Is there a way to trust the self signed cert that is being used in the vault service from this broker ?

Successfully installed and configured vault-service-broker. I was able to create a service and bind it to an application. The application was able to successfully authenticate and retrieve secrets from its space. At some point the service broker went down. When I tried to restart the broker, the broker's token had expired. Created a new token and restaged. The broker now crashes shortly startup after it fails to renew a token. I haven't been able to locate the token via the token's accessor. How do I remove the bad token or correct the permission?

The broker's log:

2017-03-22T11:35:41.54-0400 [CELL/0]     OUT Creating container
2017-03-22T11:35:41.90-0400 [CELL/0]     OUT Successfully created container
2017-03-22T11:35:42.61-0400 [CELL/0]     OUT Starting health monitoring of container
2017-03-22T11:35:42.67-0400 [APP/0]      OUT [INFO] starting broker
2017-03-22T11:35:42.67-0400 [APP/0]      OUT [DEBUG] creating mounts map[string]string{"cf/broker":"generic"}
2017-03-22T11:35:42.79-0400 [APP/0]      OUT [DEBUG] restoring bindings
2017-03-22T11:35:42.79-0400 [APP/0]      OUT [DEBUG] listing directory "cf/broker/"
2017-03-22T11:35:42.79-0400 [APP/0]      OUT [INFO] sleeping for 9m31s
2017-03-22T11:35:42.81-0400 [APP/0]      OUT [INFO] restoring info for instance 06431075-2718-449b-9186-3cf06e3d5c12
2017-03-22T11:35:42.83-0400 [APP/0]      OUT [DEBUG] decoding bind data from cf/broker/06431075-2718-449b-9186-3cf06e3d5c12
2017-03-22T11:35:42.83-0400 [APP/0]      OUT [DEBUG] listing directory "cf/broker/06431075-2718-449b-9186-3cf06e3d5c12/"
2017-03-22T11:35:42.84-0400 [APP/0]      OUT [INFO] restoring bind for instance 06431075-2718-449b-9186-3cf06e3d5c12 for binding dc9cd23b-0b83-4ad4-960f-3f3bf2d0d573
2017-03-22T11:35:42.84-0400 [APP/0]      OUT [DEBUG] reading bind from cf/broker/06431075-2718-449b-9186-3cf06e3d5c12/dc9cd23b-0b83-4ad4-960f-3f3bf2d0d573
2017-03-22T11:35:42.86-0400 [APP/0]      OUT [DEBUG] decoding bind data from cf/broker/06431075-2718-449b-9186-3cf06e3d5c12/dc9cd23b-0b83-4ad4-960f-3f3bf2d0d573
.
.
.
2017-03-22T11:35:43.27-0400 [APP/0]      OUT [INFO] restored 6 binds and 8 instances
2017-03-22T11:35:43.28-0400 [APP/0]      OUT [INFO] starting server on :8080
2017-03-22T11:35:44.66-0400 [CELL/0]     OUT Container became healthy
2017-03-22T11:35:47.86-0400 [APP/0]      OUT [DEBUG] renewing token with accessor 81199bc3-22b6-0337-3c56-b66fd8c3a03b
2017-03-22T11:35:47.90-0400 [APP/0]      OUT [ERR] token renewal failed: Error making API request.
2017-03-22T11:35:47.90-0400 [APP/0]      OUT URL: PUT https://vault.dev:443/v1/auth/token/renew
2017-03-22T11:35:47.90-0400 [APP/0]      OUT Code: 403. Errors:
2017-03-22T11:35:47.90-0400 [APP/0]      OUT * permission denied
2017-03-22T11:35:47.90-0400 [APP/0]      OUT Exit status 1
2017-03-22T11:35:47.91-0400 [CELL/0]     OUT Exit status 0
2017-03-22T11:35:47.94-0400 [CELL/0]     OUT Destroying container
2017-03-22T11:35:47.95-0400 [API/1]      OUT App instance exited with guid f1e936a3-194e-4002-95d0-796045e4f101 payload: {"instance"=>"", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"2 error(s) occurred:\n\n* 2 error(s) occurred:\n\n* Exited with status 1\n* cancelled\n* cancelled", "crash_count"=>7, "crash_timestamp"=>1490196947934464113, "version"=>"9ef948c3-b7b6-4d46-b273-ff95054d64ba"}
2017-03-22T11:35:48.37-0400 [CELL/0]     OUT Successfully destroyed container

Like generic or transit backends , do you think the PKI backend should also be established with mounts and policies on the servers and the same be populated on the VCAP_SERVICES ?

Hi there - I was just checking this out and it seems like an important part is missing. Is there a simple example of how to use the service broker in a application? Spring maybe?

I'm not terribly familiar with Vault yet so please forgive if I'm missing something

I created vault service broker and it is working correctly. I'm using the sample policy in the readme doc for the broker token.

I created a service successfully and bound it to my app, and was able to write secrets to the generic space backend and read them from my app (python app using hvac).

It appears my token has expired and I can no longer auth with it.

When I attempt to unbind my service, I get the following

cf unbind-service appname vaultservice
Unbinding app appname from service vaultservice in org org / space space as [email protected]...
Unexpected Response
Response code: 502
CC code:       10001
CC error code: CF-ServiceBrokerBadResponse
Request ID:    63df3eda-87f3-4e17-7369-ddf1e030d506
Request ID:    0c1b0c37-6b67-4c66-6df2-f44ded7d4eaf::f55eefdf-895c-4e3e-8e15-f84f3ab88eee
Description:   Service instance vault: Service broker error: failed to revoke accessor 12312b67-3693-ac35-eb52-1c515cac547b: Error making API request.

URL: POST https://vault.domain.com:8200/v1/auth/token/revoke-accessor
Code: 400. Errors:

* 1 error(s) occurred:

* invalid accessor
FAILED

Here are logs from vault broker

2017-05-17T12:57:42.97-0700 [APP/0] OUT [INFO] unbinding service df0fe535-1143-4d24-9194-655bd2eee8ea for instance b8759394-5382-4c6c-8a00-4736d75cff18
2017-05-17T12:57:42.97-0700 [APP/0] OUT [DEBUG] reading cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.00-0700 [APP/0] OUT [DEBUG] decoding binding info for cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.00-0700 [APP/0] OUT [DEBUG] revoking accessor 29912b67-3693-ac35-eb52-1c515cac547b for path cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.01-0700 [RTR/4] OUT vault-broker.apps.domain.com - [17/05/2017:19:57:42.973 +0000] "DELETE /v2/service_instances/b8759394-5382-4c6c-8a00-4736d75cff18/service_bindings/df0fe535-1143-4d24-9194-655bd2eee8ea?plan_id=0654695e-0760-a1d4-1cad-5dd87b75ed99.shared&service_id=0654695e-0760-a1d4-1cad-5dd87b75ed99 HTTP/1.1" 500 0 260 "-" "HTTPClient/1.0 (2.7.1, ruby 2.2.4 (2015-12-16))" [REDACTED]:55945 x_forwarded_for:"[REDACTED], [REDACTED]" x_forwarded_proto:"https" vcap_request_id:fffdb4f3-8487-4ec3-7413-334e9b87a726 response_time:0.041498743 app_id:564d39aa-a080-4b28-a5f3-f862109ab73e
2017-05-17T12:57:43.01-0700 [RTR/4] OUT
2017-05-17T12:57:43.16-0700 [APP/0] OUT [INFO] unbinding service df0fe535-1143-4d24-9194-655bd2eee8ea for instance b8759394-5382-4c6c-8a00-4736d75cff18
2017-05-17T12:57:43.16-0700 [APP/0] OUT [DEBUG] reading cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.17-0700 [APP/0] OUT [DEBUG] decoding binding info for cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.18-0700 [APP/0] OUT [DEBUG] revoking accessor 29912b67-3693-ac35-eb52-1c515cac547b for path cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.18-0700 [RTR/3] OUT vault-broker.apps.domain.com - [17/05/2017:19:57:43.160 +0000] "DELETE /v2/service_instances/b8759394-5382-4c6c-8a00-4736d75cff18/service_bindings/df0fe535-1143-4d24-9194-655bd2eee8ea?plan_id=0654695e-0760-a1d4-1cad-5dd87b75ed99.shared&service_id=0654695e-0760-a1d4-1cad-5dd87b75ed99 HTTP/1.1" 500 0 260 "-" "HTTPClient/1.0 (2.7.1, ruby 2.2.4 (2015-12-16))" [REDACTED]:53419 x_forwarded_for:"[REDACTED], [REDACTED]" x_forwarded_proto:"https" vcap_request_id:3486f6f0-eadd-4fe9-533a-f3614ae0dcd8 response_time:0.026972471 app_id:564d39aa-a080-4b28-a5f3-f862109ab73e
2017-05-17T12:57:43.18-0700 [RTR/3] OUT
2017-05-17T12:57:43.34-0700 [APP/0] OUT [INFO] unbinding service df0fe535-1143-4d24-9194-655bd2eee8ea for instance b8759394-5382-4c6c-8a00-4736d75cff18
2017-05-17T12:57:43.34-0700 [APP/0] OUT [DEBUG] reading cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.36-0700 [APP/0] OUT [DEBUG] decoding binding info for cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.36-0700 [APP/0] OUT [DEBUG] revoking accessor 29912b67-3693-ac35-eb52-1c515cac547b for path cf/broker/b8759394-5382-4c6c-8a00-4736d75cff18/df0fe535-1143-4d24-9194-655bd2eee8ea
2017-05-17T12:57:43.37-0700 [RTR/1] OUT vault-broker.domain.com - [17/05/2017:19:57:43.336 +0000] "DELETE /v2/service_instances/b8759394-5382-4c6c-8a00-4736d75cff18/service_bindings/df0fe535-1143-4d24-9194-655bd2eee8ea?plan_id=0654695e-0760-a1d4-1cad-5dd87b75ed99.shared&service_id=0654695e-0760-a1d4-1cad-5dd87b75ed99 HTTP/1.1" 500 0 260 "-" "HTTPClient/1.0 (2.7.1, ruby 2.2.4 (2015-12-16))" [REDACTED]:59200 x_forwarded_for:"[REDACTED], [REDACTED]" x_forwarded_proto:"https" vcap_request_id:75a78298-5d81-4215-6e93-f22298f9d440 response_time:0.03506014 app_id:564d39aa-a080-4b28-a5f3-f862109ab73e

I've been having a play with the Vault Service Broker and I've successfully deployed it and bound 2 apps deployed on my CF to a Vault service (dev mode, also running on CF).

I was expecting that the apps I bound would each have a token that would be assigned its own policy (albeit with the same contents) , but instead the tokens were both assigned the same policy i.e. there is only 1 broker policy listed in Vault policies.

My question is, is this expected behaviour?

The reason I was hoping the apps would get unique policies would be so that I could assign specific capabilities to different apps e.g. any app can encrypt, only appB can decrypt, or only appA can sign but any app can verify etc. That doesn't seem possible (at least via the broker) with what I am seeing as I can't assign a new policy to an existing token, only change a policy already associated with a token (as you know this is a feature of Vault), which doesn't allow me to give different apps different capabilities as they all use the same policy.

Hopefully I am missing something?

As a vault service broker user, I'd like to define policies in a flexible manner using human readable names. It would be very useful if the broker could use the org and space names instead of GUIDs for the paths.

Org path: /cf/foundation-1/organization-2
Space path: /cf/foundation-1/organization-2/space-1
Service ID: /cf/foundation-1/organization-2/space-1/service-name

This way I could define a policy like

path "/cf/foundation-1/organization-2/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

then assign that policy to the user group managing those secrets

vault write auth/ldap/groups/ORG-2-OPS policies=ORG-2-OPS

Application was not working due to a permission issue with the token (Renewable:false) so we wanted to stop the app and delete it and create it again.
Only way out is purge-service-instance

C:\Projects\HashicorpVaultImpl>cf delete HashicorpVaultImpl -r

Really delete the app HashicorpVaultImpl?> y
Deleting app HashicorpVaultImpl in org Test / space testspace as gundave...
FAILED
Server error, status code: 500, error code: 10011, message: Database error

C:\Projects\HashicorpVaultImpl>cf us HashicorpVaultImpl vault
Unbinding app HashicorpVaultImpl from service vault in org Test / space testspac
e as gundave...
FAILED
Service instance vault: Service broker error: failed to revoke accessor 156bf284
-0c21-7a51-424f-0aeff8a9110a: Error making API request.

URL: POST https://vault.manulife.com:8200/v1/auth/token/revoke-accessor
Code: 400. Errors:

• 1 error(s) occurred:

• invalid accessor

C:\Projects\HashicorpVaultImpl>cf unbind-service HashicorpVaultImpl vault
Unbinding app HashicorpVaultImpl from service vault in org Test / space testspac
e as gundave...
FAILED
Service instance vault: Service broker error: failed to revoke accessor 156bf284
-0c21-7a51-424f-0aeff8a9110a: Error making API request.

URL: POST https://vault.manulife.com:8200/v1/auth/token/revoke-accessor
Code: 400. Errors:

• 1 error(s) occurred:

• invalid accessor

Hey everyone

First of i was wondering if this Broker is still maintained, for example #51 mentioned, that the policy path is not correct anymore and needs to be changed, which is true and would have helped the project or new users to successfully deploy the broker.

I've deployed the broker which works to the point of creating a service and deploying an app with a service-binding, but when i either unbind an app from the service or delete the service instance the mounts in Vault still persist and do not get deleted. The only mounts in Vault that get deleted are the once from the Service itself when deleted, but a Service Unbind or deletion of an App which was bound to vault does not delete the mounts in vault.

I could only see the log's on the brokers deprovision function in broker.go for the GUID of the Service in CF, the all the Apps GUID's still stay in vault. This behaviour would end up in a situation where Vault would persist lots of Secret Path's from App's that Customers deleted or unbound from the service which i think shouldn't be.

The readme states that those mounts should get deleted.

Any info about the maintenance state of this project and maybe the problem mentioned above would be greatly appreciated.

Thanks in advance

Workflow Name: test
Branch: main
Run URL: https://github.com/hashicorp/vault-service-broker/actions/runs/4176707836

save-state deprecation warnings: 0
set-output deprecation warnings: 1
node12 deprecation warnings: 2

Please review these deprecation warnings as soon as possible and merge in the necessary updates.

GitHub will be removing support for these commands and plan to fully disable them on 31st May 2023. At this time, any workflow that still utilizes these commands will fail. See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/.

GitHub have not finalized a date for deprecating node12 yet but have indicated that this will be summer 2023. So it is advised to switch to node16 asap. See https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

If you need any help, please reach out to us in #team-rel-eng.

Hi there! I'm following the instructions for starting up the service broker, and I set up a local vault server running at http://127.0.0.1:8200 with a consul backend. I unsealed it and created an periodic orphan token with its lease set to 30 minutes. I set its policy to the .hcl file with the following policies:

# Manage internal state under "/broker", but since this token is going to 
# generate children, it needs full management of the "/cf/*" space 
path "/cf/" {
 capabilities = ["list"]
}

path "/cf/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

# List all mounts
path "sys/mounts" {
  capabilities = ["read", "list"]
}

# Create mounts under the "/cf/" prefix
path "sys/mounts/cf/*" {
  capabilities = ["create", "update", "delete"]
}

# Create policies with the "cf-*" prefix
path "sys/policy/cf-*" {
  capabilities = ["create", "update", "delete"]
}

# Create token role
path "/auth/token/roles/cf-*" {
  capabilities = ["create", "update", "delete"]
}

# Create tokens from role
path "/auth/token/create/cf-*" {
  capabilities = ["create", "update"]
}

# Revoke tokens by accessor
path "/auth/token/revoke-accessor" {
  capabilities = ["create", "update"]
}

I then set the set the Bluemix apps VAULT_ADDR to http://127.0.0.1:8200 and VAULT_TOKEN to the orphan token with the above policy. However, when running the app, im getting this error during staging:

2017-03-02T14:37:42.37-0500 [STG/0]      OUT github.com/hashicorp/cf-vault-service-broker
2017-03-02T14:38:00.03-0500 [STG/0]      OUT Uploading droplet, build artifacts cache...
2017-03-02T14:38:00.03-0500 [STG/0]      OUT Uploading build artifacts cache...
2017-03-02T14:38:18.34-0500 [STG/0]      OUT Uploading complete
2017-03-02T14:38:19.63-0500 [CELL/0]     OUT Creating container
2017-03-02T14:38:21.08-0500 [CELL/0]     OUT Successfully created container
2017-03-02T14:38:21.49-0500 [CELL/0]     OUT Starting health monitoring of container
2017-03-02T14:38:21.62-0500 [APP/0]      OUT [INFO] starting broker
2017-03-02T14:38:21.62-0500 [APP/0]      OUT [ERR] failed to start broker: failed to create mounts: Get http://127.0.0.1:8200/v1/sys/mounts: dial tcp 127.0.0.1:8200: getsockopt: connection refused
2017-03-02T14:38:21.63-0500 [APP/0]      OUT Exit status 1
2017-03-02T14:38:21.68-0500 [CELL/0]     OUT Destroying container
2017-03-02T14:38:21.70-0500 [API/0]      OUT App instance exited with guid 9679cce7-8682-453a-96ff-fda303724259 payload: {"instance"=>"", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"2 error(s) occurred:\n\n* 2 error(s) occurred:\n\n* Exited with status 1\n* cancelled\n* cancelled", "crash_count"=>1, "crash_timestamp"=>1488483501667341551, "version"=>"06447cdf-0339-4f38-a587-8b6aae960da0"}
2017-03-02T14:38:22.65-0500 [CELL/0]     OUT Successfully destroyed container
2017-03-02T14:38:23.30-0500 [STG/0]      OUT Successfully destroyed container
2017-03-02T14:38:24.30-0500 [APP/0]      OUT [INFO] starting broker
2017-03-02T14:38:24.30-0500 [APP/0]      OUT [DEBUG] creating mounts map[string]string{"cf/broker":"generic"}
2017-03-02T14:38:24.30-0500 [APP/0]      OUT [ERR] renew-token: error renewing vault token: Get http://127.0.0.1:8200/v1/auth/token/lookup-self: dial tcp 127.0.0.1:8200: getsockopt: connection refused
2017-03-02T14:38:24.30-0500 [APP/0]      OUT [ERR] failed to start broker: failed to create mounts: Get http://127.0.0.1:8200/v1/sys/mounts: dial tcp 127.0.0.1:8200: getsockopt: connection refused
2017-03-02T14:38:24.32-0500 [CELL/0]     OUT Exit status 0
2017-03-02T14:38:24.35-0500 [CELL/0]     OUT Destroying container
2017-03-02T14:38:24.38-0500 [API/10]     OUT App instance exited with guid 9679cce7-8682-453a-96ff-fda303724259 payload: {"instance"=>"", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"2 error(s) occurred:\n\n* 2 error(s) occurred:\n\n* Exited with status 1\n* cancelled\n* cancelled", "crash_count"=>2, "crash_timestamp"=>1488483504344390695, "version"=>"06447cdf-0339-4f38-a587-8b6aae960da0"}
2017-03-02T14:38:24.92-0500 [CELL/0]     OUT Creating container
2017-03-02T14:38:26.15-0500 [CELL/0]     OUT Successfully destroyed container
2017-03-02T14:38:30.29-0500 [CELL/0]     OUT Successfully created container
2017-03-02T14:38:30.80-0500 [CELL/0]     OUT Starting health monitoring of container
2017-03-02T14:38:31.04-0500 [APP/0]      OUT [INFO] starting broker
2017-03-02T14:38:31.06-0500 [APP/0]      OUT [ERR] renew-token: error renewing vault token: Get http://127.0.0.1:8200/v1/auth/token/lookup-self: dial tcp 127.0.0.1:8200: getsockopt: connection refused
2017-03-02T14:38:31.07-0500 [APP/0]      ERR panic: runtime error: invalid memory address or nil pointer dereference
2017-03-02T14:38:31.07-0500 [APP/0]      ERR [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x409d4b]
2017-03-02T14:38:31.07-0500 [APP/0]      ERR panic(0x7a2640, 0xc42000c0a0)
2017-03-02T14:38:31.07-0500 [APP/0]      ERR 	/tmp/cache/go1.7.5/go/src/runtime/panic.go:500 +0x1a1
2017-03-02T14:38:31.07-0500 [APP/0]      ERR 	/tmp/tmp.RJYH8vxKkf/.go/src/github.com/hashicorp/cf-vault-service-broker/broker.go:711 +0x17b
2017-03-02T14:38:31.07-0500 [APP/0]      ERR created by main.(*Broker).Start
2017-03-02T14:38:31.07-0500 [APP/0]      ERR 	/tmp/tmp.RJYH8vxKkf/.go/src/github.com/hashicorp/cf-vault-service-broker/broker.go:102 +0xb88
2017-03-02T14:38:31.08-0500 [APP/0]      OUT Exit status 2
2017-03-02T14:38:31.14-0500 [CELL/0]     OUT Exit status 143
2017-03-02T14:38:31.15-0500 [CELL/0]     OUT Destroying container
2017-03-02T14:38:31.20-0500 [API/6]      OUT App instance exited with guid 9679cce7-8682-453a-96ff-fda303724259 payload: {"instance"=>"", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"2 error(s) occurred:\n\n* 2 error(s) occurred:\n\n* Exited with status 2\n* cancelled\n* cancelled", "crash_count"=>3, "crash_timestamp"=>1488483511158440984, "version"=>"06447cdf-0339-4f38-a587-8b6aae960da0"}
2017-03-02T14:38:36.67-0500 [CELL/0]     OUT Successfully destroyed container

Not sure if its having an issue connecting to the local vault server. All the vault commands im running locally like initialization, seal/unseal, and mounting backends all work fine. Any advice would be appreciated!