cloudfoundry-community / consul-boshrelease Goto Github PK

BOSH release for consul

License: Other

Ruby 1.21% Shell 45.33% HTML 4.59% Perl 48.87%

consul bosh-release

consul-boshrelease's Introduction

Deploy Consul to BOSH

Hashicorp Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. It also offers flexible key/value storage.

One of the fastest ways to get consul running on any infrastructure is to deploy this BOSH release.

Discussions and CI notifications at #consul-boshrelease channel on https://slack.cloudfoundry.org

Usage

To use this bosh release, first upload it to your bosh:

export BOSH_ENVIRONMENT=<alias>
export BOSH_DEPLOYMENT=consul

git clone https://github.com/cloudfoundry-community/consul-boshrelease.git
cd consul-boshrelease
bosh deploy manifests/consul.yml -o manifests/operators/firsttime.yml

If your BOSH does not have Credhub/Config Server, then remember --vars-store to allow generation of certificates.

The consul.yml manifest is deliberately missing the required update: section of the manifest. This is to ensure that you - the operator - choose the correct update: section - either firsttime.yml for the first deployment (deploy all instances at the same time so they form a cluster) or existing.yml for all subsequent deployments (rolling updates).

If you get the following error then you have forgotten to provide either of these two operator files:

Task 1045 | 23:24:04 | Preparing deployment: Preparing deployment (00:00:00)
                     L Error: Required property 'update' was not specified in object ({"instance_groups"=>[{"azs"=>["z1", "z2", "z3"], "instances"=>3, "jobs"=>...

Subsequent deploys/upgrades

Replace manifests/operators/firsttime.yml above with manifests/operators/existing.yml so that each instance is updated one at a time:

bosh deploy manifests/consul.yml -o manifests/operators/existing.yml

consul-boshrelease's People

Contributors

Stargazers

Watchers

Forkers

philipz devcamcar jbayer dpb587 nimbus-cloud hmcgonig natfitz predix cnelson jmcarp hkumarmk shankj3 kitsirota scottillogical ships jvikes11 bohem1 alphagov

consul-boshrelease's Issues

Deploying consul servers do not rejoin each other

    2014/05/16 18:49:38 [INFO] serf: EventMemberLeave: consul-warden-consul_bootstrap_z1-0 10.244.4.2
    2014/05/16 18:49:38 [INFO] serf: EventMemberLeave: consul-warden-consul_server_z1-0 10.244.4.6
    2014/05/16 18:49:38 [INFO] consul: member 'consul-warden-consul_bootstrap_z1-0' left, deregistering
    2014/05/16 18:49:38 [INFO] raft: Removed peer 10.244.4.2:8300, stopping replication
    2014/05/16 18:49:38 [INFO] consul: member 'consul-warden-consul_server_z1-0' left, deregistering
    2014/05/16 18:49:38 [INFO] raft: Removed peer 10.244.4.6:8300, stopping replication

# consul members
consul-warden-consul_server_z1-1         10.244.4.10:8301  alive  role=consul,dc=dc1,vsn=1,vsn_min=1,vsn_max=1,port=8300
consul-warden-consul_server_z1-0         10.244.4.6:8301   left   role=consul,dc=dc1,vsn=1,vsn_min=1,vsn_max=1,port=8300
consul-warden-consul_bootstrap_z1-0      10.244.4.2:8301   left   role=consul,dc=dc1,vsn=1,vsn_min=1,vsn_max=1,port=8300,bootstrap=1
redis-warden-redis_leader_z1-0           10.244.2.6:8301   alive  role=node,dc=dc1,vsn=1,vsn_min=1,vsn_max=1
redis-warden-redis_z1-0                  10.244.2.10:8301  alive  role=node,dc=dc1,vsn=1,vsn_min=1,vsn_max=1
redis-counter-warden-redis_counter_z1-0  10.244.3.6:8301   alive  role=node,dc=dc1,vsn=1,vsn_min=1,vsn_max=1

consul bosh release vm resurrectes with new Ip uses old advertise_addr

I trying cluster with 3 node .each node has vault/consul.I am able to setup cluster.For testing purpose I deleted Leader VM.
Node 1 192.168.0.158 Leader "advertise_addr": "192.168.0.158"
Node 2 192.168.0.157 Follower
Node 3 192.168.0.156 Follower

I deleted Node 1 (Leader) using Nova delete command.After 5 minutes it resurrects again.

bosh vms shows ip as 192.168.0.154, but advertise_addr still uses old ip 192.168.0.158

I checked agent.json.erb file, looks like folloswing line returns ip address
my_ip = spec.networks.send(spec.networks.methods(false).first).ip

agent.json
"advertise_addr": "192.168.0.158",

What is going wrong here ?

[aws] Can't deploy

Followed the readme, got this:

Director task 1989
Deprecation: Ignoring cloud config. Manifest contains 'networks' section.

  Started preparing deployment > Preparing deployment. Done (00:00:00)

  Started preparing package compilation > Finding packages to compile. Done (00:00:00)

  Started compiling packages
  Started compiling packages > envconsul/90d4cc3b4e290c3833cf5e32d0b5c99f4a63c0be
  Started compiling packages > consul-template/561a4a5d99c375822876d5482ed24f790a0e216b
  Started compiling packages > consul/30f12d1e70d89f28b34a433d2b885a03ae41adae
   Failed compiling packages > envconsul/90d4cc3b4e290c3833cf5e32d0b5c99f4a63c0be: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected params[:filters][0][:values][0] to be a String, got value nil (class: NilClass) instead.'' in 'create_vm' CPI method (00:00:10)
   Failed compiling packages > consul/30f12d1e70d89f28b34a433d2b885a03ae41adae: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected params[:filters][0][:values][0] to be a String, got value nil (class: NilClass) instead.'' in 'create_vm' CPI method (00:00:11)
   Failed compiling packages > consul-template/561a4a5d99c375822876d5482ed24f790a0e216b: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected params[:filters][0][:values][0] to be a String, got value nil (class: NilClass) instead.'' in 'create_vm' CPI method (00:00:11)
   Failed compiling packages (00:00:11)

Error 100: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected params[:filters][0][:values][0] to be a String, got value nil (class: NilClass) instead.'' in 'create_vm' CPI method

Bosh v2 links

Moving to links based setup will simplify deployment manifests.

SSL Cert Errors

Hello,

It deployed to AWS, and I generated the creds.yml with with bosh interpolate.

However, something seems wrong with the certs:

# curl https://127.0.0.1:8500/v1/agent/members -k
curl: (35) error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate

From the browser (Chrome):

This site can’t provide a secure connection
172.16.10.6 didn’t accept your login certificate, or one may not have been provided.
Try contacting the system admin.
ERR_BAD_SSL_CLIENT_AUTH_CERT

Remove `encryption_key` from final.yml

Newer BOSH CLI's don't honor the flag anymore, and it makes it impossible to sync blobs:

→  bosh create release --force
Syncing blobs...
envconsul/envconsul_0.3.0_linux_amd64.tar... downloading 1.9M (600%)
/Users/jhunt/.rvm/gems/ruby-2.2.1/gems/blobstore_client-1.3157.0/lib/blobstore_client/sha1_verifiable_blobstore_client.rb:38:in `check_sha1': sha1 mismatch expected=14d879bfcbbbd7025193dee981dadf8858eaa669 actual=ba3c2948cf1442e774b2f2589500afd789c085bb (Bosh::Blobstore::BlobstoreError)
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/blobstore_client-1.3157.0/lib/blobstore_client/sha1_verifiable_blobstore_client.rb:24:in `get'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/blobstore_client-1.3157.0/lib/blobstore_client/retryable_blobstore_client.rb:19:in `block in get'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/retryable.rb:28:in `call'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/retryable.rb:28:in `block in retryer'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/retryable.rb:26:in `loop'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/retryable.rb:26:in `retryer'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/blobstore_client-1.3157.0/lib/blobstore_client/retryable_blobstore_client.rb:18:in `get'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_cli-1.3157.0/lib/cli/blob_manager.rb:316:in `download_blob'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_cli-1.3157.0/lib/cli/blob_manager.rb:242:in `block (3 levels) in process_index'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/thread_pool.rb:77:in `call'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/thread_pool.rb:77:in `block (2 levels) in create_thread'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/thread_pool.rb:63:in `loop'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/bosh_common-1.3157.0/lib/common/thread_pool.rb:63:in `block in create_thread'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
    from /Users/jhunt/.rvm/gems/ruby-2.2.1/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'

recursor should default to public DNS

For bosh-lite (which doesn't provide BOSH DNS) the default nameserver is not a useful DNS recursor. Try 8.8.8.8 or something.

Access Denied trying to `bosh upload` per README

When I run the bosh upload command in the README, pulling down from S3:

bosh upload release https://consul-boshrelease.s3.amazonaws.com/boshrelease-consul-7.tgz

I get an access denied. Curling the endpoint shows that S3 is denying public / anon access:

→  curl -v https://consul-boshrelease.s3.amazonaws.com/boshrelease-consul-7.tgz
*   Trying 54.231.81.56...
* Connected to consul-boshrelease.s3.amazonaws.com (54.231.81.56) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.s3.amazonaws.com
* Server certificate: DigiCert SHA2 High Assurance Server CA
* Server certificate: DigiCert High Assurance EV Root CA
> GET /boshrelease-consul-7.tgz HTTP/1.1
> Host: consul-boshrelease.s3.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< x-amz-request-id: 149FB58E4245F56B
< x-amz-id-2: I06ta3cCAzBzaZDpkoFDTIcW7QLGGJFJOSoM2XEwKl2CeB3OJhOlKwJfS1nLyWogayWtnnjIeRU=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Wed, 13 Jan 2016 16:29:38 GMT
< Server: AmazonS3
<
<?xml version="1.0" encoding="UTF-8"?>
* Connection #0 to host consul-boshrelease.s3.amazonaws.com left intact
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>149FB58E4245F56B</RequestId><HostId>I06ta3cCAzBzaZDpkoFDTIcW7QLGGJFJOSoM2XEwKl2CeB3OJhOlKwJfS1nLyWogayWtnnjIeRU=</HostId></Error>

Consul agent needs to work nicely with other nameservers

See hashicorp/consul#228

License?

Can you commit and declare an official license for this repo?

Upgrade consul

Consul 0.7.1 is out and includes some bug fixes: https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#071-november-10-2016, plus more in https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#070-september-14-2016. At least one of these bugs has bitten us--any objection to upgrading?

cc @cnelson

Allow Explicit Service Registration

If other jobs define a /var/vcap/jobs/*/consul directory, we pick that up automagically, but if they don't (as is the case with all Diego components), we need a way to inject service definitions easily.

Suggest a new property, consul.services that defines a map of service name -> definitions.

Correct README.md formatting

The most recent commit messed up the formatting for README.md, starting at the Usage section. It looks like the edits in the Installation section are the cause of the problem.

Need for consul agent ui set to false by default

https://github.com/cloudfoundry-community/consul-boshrelease/blob/master/jobs/consul/templates/consul/agent.json.erb#L16 can we have this property configurable?

ui: true, in consul agent erb file.

It should be false by default. Right now, my understanding is UI of consul exposes the datastore. If Vault is using Consul as backend and Vault has UI disabled, we would like to have consul UI to be disabled as well. With hard-coded config property in agent.json.erb it's difficult right now to make this change.

Any support path for garden

Are there any plans to provide support for garden based containers and future support cloud config?

Consul wont start after adding shield-agent template

When adding shield-agent template to a vm running in a consul cluster consul wont start:
consul.stderr:

Invalid trailing arguments: [agent -config-dir /var/vcap/data/consul
-config-dir /var/vcap/jobs/consul/consul
-config-dir /var/vcap/sys/run/consul/consul
-config-dir /var/vcap/sys/run/consul/shield-agent 
-config-dir /var/vcap/sys/run/consul/vault
-pid-file /var/vcap/sys/run/consul/consul.pid]

Usage: consul [global options] <verb> [verb options]

Removing the shield-agent template consistently makes it work again.

Need to advertise floating ip instead of private ip in Bosh & Openstack

Hi there,

Our consul is installed via Bosh on our Openstack machines and consequently thing they're running on 192.168.1.x (example IP) which is what they publish to agents. The consul servers also have floating IPs from Openstack in the 192.168.2.x range.

We have some agents on 192.168.2.x which we want to connect to the servers, however, the servers are advertising the "wrong" ip addresses. Namely 192.168.1.x

How can I get them to "advertise" the 192.168.2.x IPs using this Bosh release?!?

Thanks, Martijn

Allow configuration of peer addresses

When this boshrelease moved to links, the ability to customize peer addrs was completely removed. This means we cant hook up a consul cluster across multiple bosh directors anymore. Would be nice if that was once again supported.

bosh2 manifest persistent disk type change

Suggest changing persistent disk type from 1GB to default in manifests/consul.yml (line12)
from persistent_disk_type: 1GB to persistent_disk_type: default

When I deployed this into my vbox BOSH it failed because of a missing type 1GB as I have not defined this in my cloud config.
I do have a default persistent disk type defined in my vbox BOSH cloud config as a lot of other cloudfoundry community BOSH releases specify default for persistent disk type in their example manifests.

Upgrade to latest consul

https://github.com/starkandwayne/hashicorp-release-resource is a new concourse resource to watch/download latest versions

maintenance status of this release

I am interested in fulfilling #43 and have the ability now to do so, but want to make sure I do not clobber anybody with active interest in this release.

i see that no updates have come through in some time, as CF has moved away from Consul, but i am interested in it in ancillary projects without CF :)

Anybody else working on or using this release actively?

Drop golang package

All packages use precompiled binaries for the Linux platform, so it isn't actually used anywhere.

Error starting agent: agent: timeout starting DNS servers

Hello,

I've been trying to deploy consul-boshrelease in my local BOSH environment. However, when I issue the command bosh vms I see the following result:

Task 135. Done

Deployment 'consul'

Instance                                     Process State  AZ  IPs          VM CID                                  VM Type  Active
consul/7432a2b8-cd63-4ad2-a7f2-8dd328c54b5a  running        z3  10.244.0.32  c-c8e17a7c-707c-408a-64ea-4fe548ab2846  default  true
consul/bd8fcd0f-65df-46a2-b9e4-f96b4c695a42  failing        z1  10.244.0.25  c-3f9f1e0c-c040-41eb-4c21-86c4a2049ff7  default  true
consul/c5511d3c-b8b1-4f14-b842-c9dc0b1868e0  running        z2  10.244.0.31  c-bb41ce30-36d0-48aa-425d-b8fae8f9ed98  default  true

When I log to the failing machine, I see the following repeating entries for the consul job:

==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: agent: timeout starting DNS servers
==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: agent: timeout starting DNS servers
==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: agent: timeout starting DNS servers

I was not able to figure out what the problem is, so I searched for the error message in the consul project and I found the message here:

https://github.com/hashicorp/consul/blob/e305443db4ba8295510faf2402e584650efeb3f8/agent/agent.go#L478

	// wait for servers to be up
	timeout := time.After(time.Second)
	for range a.config.DNSAddrs {
		select {
		case addr := <-notif:
			a.logger.Printf("[INFO] agent: Started DNS server %s (%s)", addr.String(), addr.Network())
			continue
		case <-timeout:
			return fmt.Errorf("agent: timeout starting DNS servers")
		}
}

Unfortunately, I couldn't figure out the root cause of the issue. Could you please guide me and suggest what's the probable root cause? I guess it is a configuration issue or perhaps environmental issue but at this point I don't have enough information to narrow it down. That's why I need your help.

Thank you in advance for your kind cooperation!

Regards,
Beloslava