This project will create an elasticsearch cluster in AWS using multiple availability zones. The cluster is located in a private subnet and communicates via private ip addresses.

• Terraform >= v0.6.15
• Elasticsearch IAM profile called elasticSearchNode with EC2 permissions. You can also use the iam project found in the iam directory. This only needs to be done once per account.

Packer AMI's

We use prebuild Packer AMI's built from these projects:

• packer-elasticsearch

• install Terraform and add it to your PATH.
• clone this repo.
• terraform get

We rely on AWS credentials to have been set elsewhere, for example using environment variables. We also use terraform_exec to execute terraform that saves environment state to S3.

aws kms encrypt --key-id --plaintext fileb://<(echo ) --output text --query CiphertextBlob | base64 | base64 -d

This is then provided to terraform via encrypted_atlas_token.

Create a configuration file such as ~/.aws/default.tfvars which can include mandatory and optional variables such as:

key_name="<key name>"

stream_tag="<used for aws resource groups>"

aws_region="ap-southeast-2"
ami="ami-7ff38945"

vpc_id="xxx"
additional_security_groups=""

es_cluster="cluster name"
es_environment="dev"
volume_name="/dev/sdh"
volume_size="10"

instances="3"
availability_zones="ap-southeast-2a,ap-southeast-2b"
subnets="subnet-xxxxx,subnet-yyyyy"

# consul variables
dns_server  = "172.100.0.2"
consul_dc   = "dc0"
atlas       = "atlas user"
atlas_token = "atlas token"
# internal hosted zone

These variables can also be overriden when running terraform like so:

terraform (plan|apply|destroy) -var 'ami=foozie'

The variables.tf terraform file can be further modified, for example it defaults to ap-southeast-2 for the AWS region.

Execute the plan to see if everything works as expected.

terraform plan -var-file ~/.aws/default.tfvars -state='environment/development.tfstate'

If all looks good, lets build our infrastructure!

terraform apply -var-file ~/.aws/default.tfvars -state='environment/development.tfstate'

A security group is created using terraform that opens up Elasticsearch and ssh ports. We can also add extra pre-existing security groups to our Elasticsearch instances like so:

terraform plan -var-file '~/.aws/default.tfvars' -var 'additional_security_groups=sg-xxxx, sg-yyyy'

• Update this readme