Chef Provisioning was officially end-of-life on August 31, 2019 and is no longer included with Chef DK. The Chef Provisioning source code and drivers have been moved into the chef-boneyard organization. If you are a current user of Chef Provisioning, please contact your Chef Customer Success Manager or Account Representative to review your options.
This README is a work in progress. Feel free to open PRs expanding it!
There are 3 ways you can provide your AWS Credentials. We will look for credentials in the order from below and use the first one found. This precedence order is taken from http://docs.aws.amazon.com/sdkforruby/api/index.html#Configuration:
- Through the environment variables
ENV["AWS_ACCESS_KEY_ID"],ENV["AWS_SECRET_ACCESS_KEY"]and optionallyENV["AWS_SESSION_TOKEN"] - The shared credentials ini file. The default location is
~/.aws/credentialsbut you can overwrite this by specifyingENV["AWS_CONFIG_FILE"]. You can specify multiple profiles in this file and select one with theENV["AWS_DEFAULT_PROFILE"]environment variable or via the driver url. For example, a driver url ofaws:staging:us-east-1would use the profilestaging. If you do not specify a profile then thedefaultone is used. Read this for more information about profiles. - From an instance profile when running on EC2. This accesses the local metadata service to discover the local instance's IAM instance profile.
When using machine_batch with a large number of machines it is possible to overwhelm the AWS SDK until it starts returning AWS::EC2::Errors::RequestLimitExceeded. You can configure the AWS SDK to retry these errors automatically by specifying
chef_provisioning({:aws_retry_limit => 10})in your client.rb for the provisioning workstation. The default :aws_retry_limit is 5.
By default, the time we will wait for a machine to become ready or for the transport to become ready is 120 seconds (each).
For a machine_image we wait 300 seconds for the AMI to be created. These timeouts can be configured with
chef_provisioning({:image_max_wait_time => 600, :machine_max_wait_time => 240})in your client.rb for the provisioning workstation.
TODO: List out weird/unique things about resources here. We don't need to document every resource because users can look at the resource model.
You can specify an existing key pair to upload by specifying the following:
aws_key_pair 'my-aws-key' do
private_key_path "~boiardi/.ssh/my-aws-key.pem"
public_key_path "~boiardi/.ssh/my-aws-key.pub"
allow_overwrite false # Set to true if you want to regenerate this each chef run
endIn the AWS SDK V1, you must specify key_pair instead of key_name when specifying the key name to use for machines in the auto scaling group. This is fixed in V2 and uses key_name like machines do.
aws_launch_configuration 'example-windows-launch-configuration' do
image 'example-windows-image'
instance_type 't2.medium'
options security_groups: 'example-windows-sg',
key_pair: 'my-key-name',
ebs_optimized: false,
detailed_instance_monitoring: false,
iam_instance_profile: 'example-windows-role',
user_data: <<-EOF
<powershell>
# custom powershell code goes here, executed at instance creation time
</powershell>
EOF
endIf you specify internet_gateway true the VPC will create and manage its own internet gateway.
Specifying internet_gateway false will delete that managed internet gateway.
Specifying main_routes without main_route_table will update the 'default' route table
that is created when AWS creates the VPC.
Specifying main_route_table without specifying main_routes will update the main route
association to point to the provided route table.
If you specify both main_routes and main_route_table we will update the main_route_table
to have the specified main_routes. IE, running
aws_route_table 'ref-main-route-table' do
vpc 'ref-vpc'
routes '0.0.0.0/0' => :internet_gateway
end
aws_vpc 'ref-vpc' do
main_route_table 'ref-main-route-table'
main_routes '0.0.0.0/1' => :internet_gateway
end
aws_vpc 'ref-vpc' do
main_routes '0.0.0.0/2' => :internet_gateway
endwill cause resource flapping. The ref-main-route-table resource will set the routes to /0
and then the vpc will set the routes to /1. Then because ref-main-route-table is set
to the main route for ref-vpc the third resource will set the routes to /2.
The takeaway from this is that you should either specify main_routes on your VPC and only
manage the routes through that, OR only specify main_route_table and manage the routes
through the aws_route_table resource.
If you specify action :purge on the VPC it will attempt to delete ALL resources contained in this
VPC before deleting the actual VPC.
A potential danger of this is that it does not delete the data bag entries for tracked AWS objects.
If you :purge a VPC and it has aws_route_table[ref-route] in it, the data bag entry for
ref-route is not automatically destroyed. Purge is most useful for testing to ensure no objects
are left that AWS can charge for.
You can pass machine options that will be used by machine, machine_batch and machine_image to
configure the machine.
These options are an extension of the base options. Please see that for a list of the machine_options shared between drivers.
The full syntax available in the bootstrap_options hash is the hash expected by the AWS create_instances method. The options seen below in the example are the default options.
with_machine_options({
# See https://github.com/chef/chef-provisioning#machine-options for options shared between drivers
bootstrap_options: {
# http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Resource.html#create_instances-instance_method
# lists the available options. The below options are the default
image_id: "ami-5915e11d", # default for us-west-1
instance_type: "t2.micro",
key_name: "chef_default", # If not specified, this will be used and generated
key_path: "~/.chef/keys/chef_default", # only necessary if storing keys some other location
user_data: "...", # Only defaulted on Windows instances to start winrm
},
use_private_ip_for_ssh: false, # DEPRECATED, use `transport_address_location`
transport_address_location: :public_ip, # `:public_ip` (default), `:private_ip` or `:dns`. Defines how SSH or WinRM should find an address to communicate with the instance.
is_windows: true, # false by default
winrm_username: "Administrator",
winrm_password: "password", # Override if you have specified your own password and are not fetching the password from AWS
})This options hash can be supplied to either with_machine_options at the recipe level or directly into the machine_options
attribute.
You can configure the ELB options by setting with_load_balancer_options or specifying them on each load_balancer resource.
machine 'test1'
m2 = machine 'test2'
load_balancer "my_elb" do
machines ['test1', m2]
load_balancer_options({
subnets: subnets,
security_groups: [load_balancer_sg],
listeners: [
{
instance_port: 8080,
protocol: 'HTTP',
instance_protocol: 'HTTP',
port: 80
},
{
instance_port: 8080,
protocol: 'HTTPS',
instance_protocol: 'HTTP',
port: 443,
ssl_certificate_id: "arn:aws:iam::360965486607:server-certificate/cloudfront/foreflight-2015-07-09"
}
],
health_check: {
healthy_threshold: 2,
unhealthy_threshold: 4,
interval: 12,
timeout: 5,
target: 'HTTPS:443/_status'
}
})The available parameters for load_balancer_options can be viewed in the aws docs.
If you wish to enable sticky sessions, pass a sticky_sessions key to the
load_balancer_options and specify a cookie name and the ports that should be
sticky. In the above example, it would look like this:
machine 'test1'
m2 = machine 'test2'
load_balancer "my_elb" do
machines ['test1', m2]
load_balancer_options({
subnets: subnets,
security_groups: [load_balancer_sg],
listeners: [
{
instance_port: 8080,
protocol: 'HTTP',
instance_protocol: 'HTTP',
port: 80
},
{
instance_port: 8080,
protocol: 'HTTPS',
instance_protocol: 'HTTP',
port: 443,
ssl_certificate_id: "arn:aws:iam::360965486607:server-certificate/cloudfront/foreflight-2015-07-09"
}
],
sticky_sessions: {
cookie_name: 'my-app-cookie',
ports: [80, 443]
}
})NOTES:
-
You can specify either
ssl_certificate_idorserver_certificatein a listener but the value to both parameters should be the ARN of an existing IAM::ServerCertificate object. -
The
sticky_sessionsoption currently only supports Application-Controlled Session Stickiness.
RDS instances have many options. Some of them live as first class attributes. Any valid RDS option that is not a first class attribute can still be set via a hash in additional_options.
If you set an attribute and also specify it in additional_options, the resource will chose the attribute and not what is specified in additional_options.
To illustrate, note that the following example defines multi_az as both an attribute and in the additional_options hash:
aws_rds_instance "test-rds-instance2" do
engine "postgres"
publicly_accessible false
db_instance_class "db.t1.micro"
master_username "thechief"
master_user_password "securesecure"
multi_az false
additional_options(multi_az: true)
end
The above would result in a new aws_rds_instance with multi_az being false.
Additional values for additional_options can view viewed in the aws docs.
See this example for how to set up a DB Subnet Group and pass it to your RDS Instance.
See Pointing Boxes at Chef Servers
All resources (incuding base resources like machine) that are taggable support an aws_tags attribute which accepts a single layer hash. To set just the key of an AWS tag specify the value as nil. EG, aws_tags {my_tag_key: nil}. Some AWS objects cannot accept nil values and will automatically convert it to an empty string.
Some AWS objects (may EC2) view the Name tag as unique - it shows up in a Name column in the AWS console. By default we specify the Name tag as the resource name. This can be overridden by specifying aws_tags {Name: 'some other name'}.
You can remove all the tags except the Name tag by specifying aws_tags({}).
Tag keys and values can be specified as symbols or strings but will be converted to strings before sending to AWS.
Examples:
aws_ebs_volume 'ref-volume' do
aws_tags company: 'my_company', 'key_as_string' => :value_as_symbol
end
aws_vpc 'ref-vpc' do
aws_tags 'Name' => 'custom-vpc-name'
endTo enable tagging support you must make specific changes to the Resource and Attribute. For the Resource it needs to include the attribute aws_tags. This should be done by include Chef::Provisioning::AWSDriver::AWSTaggable on the Resource.
The AWSProvider class will automatically try to call converge_tags when running the action_create method. You should instantiate an instance of the AWSTagger and provide it a strategy depending on the client used to perform the tagging. For example, an RDS Provider should define
def aws_tagger
@aws_tagger ||= begin
rds_strategy = Chef::Provisioning::AWSDriver::TaggingStrategy::RDS.new(
new_resource.driver.rds,
construct_arn(new_resource),
new_resource.aws_tags
)
Chef::Provisioning::AWSDriver::AWSTagger.new(rds_strategy, action_handler)
end
end
def converge_tags
aws_tagger.converge_tags
endThe aws_tagger method is used by the tests to assert that the object tags are correct. These methods can be encapsulated in an module for DRY purposes, as the EC2 strategy shows.
Finally, you should add 3 standard tests for taggable objects - 1) Tags can be created on a new object, 2) Tags can be updated on an existing object with tags and 3) Tags can be cleared by setting aws_tags({}). Copy the tests from an existing spec file and modify them to support your resource. TODO make a module that copies these tests for us. Right now it is complicated by the fact that some resources have required attributes that others don't.
All chef-provisioning-aws resources have a aws_object method that will return the AWS object. The base
resources machine, machine_image and load_balancer are monkeypatched to also include the aws_object
method and should respond to it like all other resources.
The AWS object won't exist until the resource converges, however. An example of how to do this looks like:
my_vpc = aws_vpc 'my_vpc' do
cidr_block '10.0.0.0/24'
main_routes '0.0.0.0/0' => :internet_gateway
internet_gateway true
end
my_sg = aws_security_group 'my_sg' do
vpc lazy { my_vpc.aws_object.id }
inbound_rules '0.0.0.0/0' => [ 22, 80 ]
end
my_subnet = aws_subnet 'my_subnet' do
vpc lazy { my_vpc.aws_object.id }
cidr_block '10.0.0.0/24'
availability_zone 'eu-west-1a'
map_public_ip_on_launch true
end
machine 'my_machine' do
machine_options(
lazy do
{
bootstrap_options: {
subnet_id: my_subnet.aws_object.id,
security_group_ids: [my_sg.aws_object.id]
}
}
end
)
endNote the use of the lazy attribute modifier. This is necessary because when the resources are compiled
the aws_objects do not exist yet, so we must wait to reference them until the converge phase.
You have access to the aws object when necessary, but often it isn't needed. The above example is better written as:
aws_vpc 'my_vpc' do
cidr_block '10.0.0.0/24'
main_routes '0.0.0.0/0' => :internet_gateway
internet_gateway true
end
aws_security_group 'my_sg' do
vpc 'my_vpc'
inbound_rules '0.0.0.0/0' => [ 22, 80 ]
end
aws_subnet 'my_subnet' do
vpc 'my_vpc'
cidr_block '10.0.0.0/24'
availability_zone 'eu-west-1a'
map_public_ip_on_launch true
end
machine 'my_machine' do
machine_options(
bootstrap_options: {
subnet_id: 'my_subnet',
security_group_ids: ['my_sg']
}
)
endWhen specifying bootstrap_options and any attributes which reference another aws resource, we
perform lookup_options.
This tries to turn elements with names like vpc, security_group_ids, machines, launch_configurations,
load_balancers, etc. to the correct AWS object.
The behavior of the machine resource is that once a machine has been allocated,
the bootstrap_options can not be modified. This currently reflects AWS's
restrictions on EC2-Classic instances, but AWS does allow users to modify the
security groups associated with EC2-VPC instances. This is because the security
groups for EC2-VPC instances are actually associated with that instance's
Network Interface. This means that if you wish to modify the security groups
associated with an EC2-VPC instance, you'll want to use the aws_network_interface
resource.
The first step is to find the Network Interface ID (eni-XXXXXXX) associated
with your machine. You can do this by inspecting the instance details in the AWS
Console or by using the AWS CLI. If you want to use the AWS CLI, you can use
this command replacing MACHINE_NAME with the name of the machine resource
you wish to update.
aws ec2 describe-instances --filter "Name=tag:Name,Values=MACHINE_NAME" | grep NetworkInterfaceIdOnce you have the Network Interface ID, in a chef-provisioning recipe you can specify the following resource:
require 'chef/provisioning/aws_driver'
with_driver 'aws' # specify the profile / region as appropriate
aws_network_interface 'eni-XXXXXXXX' do
security_groups ['sg-XXXXXXXX', 'sg-YYYYYYYY']
endThis resource can be in the same chef-provisioning recipe as the corresponding machine resource, or it can be in a different one.
To run the integration tests execute bundle exec rspec. If you have not set it up,
you should see an error message about a missing environment variable AWS_TEST_DRIVER. You can add
this as a normal environment variable or set it for a single run with AWS_TEST_DRIVER=aws::eu-west-1 bundle exec rspec. The format should match what with_driver expects.
You will also need to have configured your ~/.aws/config or environment variables with your
AWS credentials.
This creates real objects within AWS. The tests make their best effort to delete these objects after each test finishes but errors can happen which prevent this. Be aware that this may charge you!
If you find the tests leaving behind resources during normal conditions (IE, not when there is an
unexpected exception) please file a bug. Most objects can be cleaned up by deleting the test_vpc
from within the AWS browser console.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent