Code Monkey home page Code Monkey logo

terraform-inventory's Introduction

Terraform Inventory

Build Status GitHub release GitHub release

This is a little Go app which generates a dynamic Ansible inventory from a Terraform state file. It allows one to spawn a bunch of instances with Terraform, then (re-)provision them with Ansible.

The following providers are supported:

  • AWS
  • CloudStack
  • DigitalOcean
  • Docker
  • Exoscale
  • Google Compute Engine
  • Hetzner Cloud
  • libvirt
  • Linode
  • OpenStack
  • Packet
  • ProfitBricks
  • Scaleway
  • SoftLayer
  • VMware
  • Nutanix
  • Open Telekom Cloud
  • Yandex.Cloud
  • Telmate/Proxmox

It's very simple to add support for new providers. See pull requests with the provider label for examples.

Help Wanted 🙋

This library is stable, but I've been neglecting it somewhat on account of no longer using Ansible at work. Please drop me a line if you'd be interested in helping to maintain this tool.

Installation

On OSX, install it with Homebrew:

brew install terraform-inventory

Alternatively, you can download a release suitable for your platform and unzip it. Make sure the terraform-inventory binary is executable, and you're ready to go.

Usage

If you are using remote state (or if your state file happens to be named terraform.tfstate), cd to it and run:

ansible-playbook --inventory-file=/path/to/terraform-inventory deploy/playbook.yml

This will provide the resource names and IP addresses of any instances found in the state file to Ansible, which can then be used as hosts patterns in your playbooks. For example, given for the following Terraform config:

resource "digitalocean_droplet" "my_web_server" {
  image = "centos-7-0-x64"
  name = "web-1"
  region = "nyc1"
  size = "512mb"
}

The corresponding playbook might look like:

- hosts: my_web_server
  tasks:
    - yum: name=cowsay
    - command: cowsay hello, world!

Note that the instance was identified by its resource name from the Terraform config, not its instance name from the provider. On AWS, resources are also grouped by their tags. For example:

resource "aws_instance" "my_web_server" {
  instance_type = "t2.micro"
  ami = "ami-96a818fe"
  tags = {
    Role = "web"
    Env = "dev"
  }
}

resource "aws_instance" "my_worker" {
  instance_type = "t2.micro"
  ami = "ami-96a818fe"
  tags = {
    Role = "worker"
    Env = "dev"
  }
}

Can be provisioned separately with:

- hosts: role_web
  tasks:
    - command: cowsay this is a web server!

- hosts: role_worker
  tasks:
    - command: cowsay this is a worker server!

- hosts: env_dev
  tasks:
    - command: cowsay this runs on all dev servers!

More Usage

Ansible doesn't seem to support calling a dynamic inventory script with params, so if you need to specify the location of your state file or terraform directory, set the TF_STATE environment variable before running ansible-playbook, like:

TF_STATE=deploy/terraform.tfstate ansible-playbook --inventory-file=/path/to/terraform-inventory deploy/playbook.yml

or

TF_STATE=../terraform ansible-playbook --inventory-file=/path/to/terraform-inventory deploy/playbook.yml

If TF_STATE is a file, it parses the file as json, if TF_STATE is a directory, it runs terraform state pull inside the directory, which is supports both local and remote terraform state.

It looks for state config in this order

  • TF_STATE: environment variable of where to find either a statefile or a terraform project
  • TI_TFSTATE: another environment variable similar to TF_STATE
  • terraform.tfstate: it looks in the state file in the current directory.
  • .: lastly it assumes you are at the root of a terraform project.

Alternately, if you need to do something fancier (like downloading your state file from S3 before running), you might wrap this tool with a shell script, and call that instead. Something like:

#!/bin/bash
/path/to/terraform-inventory $@ deploy/terraform.tfstate

Then run Ansible with the script as an inventory:

ansible-playbook --inventory-file=bin/inventory deploy/playbook.yml

This tool returns the public IP of the host by default. If you require the private IP of the instance to run Ansible, set the TF_KEY_NAME environment variable to private_ip before running the playbook, like:

TF_KEY_NAME=private_ip ansible-playbook --inventory-file=/path/to/terraform-inventory deploy/playbook.yml

By default, the ip address is the ansible inventory name. The TF_HOSTNAME_KEY_NAME environment variable allows you to overwrite the source of the ansible inventory name.

TF_HOSTNAME_KEY_NAME=name ansible-playbook --inventory-file=/path/to/terraform-inventory deploy/playbook.yml

Development

It's just a Go app, so the usual:

go get github.com/adammck/terraform-inventory

To test against an example statefile, run:

terraform-inventory --list fixtures/example.tfstate
terraform-inventory --host=52.7.58.202 fixtures/example.tfstate

To update the fixtures, populate fixtures/secrets.tfvars with your DO and AWS account details, and run fixtures/update. To run a tiny Ansible playbook on the example resourecs, run:

TF_STATE=fixtures/example.tfstate ansible-playbook --inventory-file=/path/to/terraform-inventory fixtures/playbook.yml

You almost certainly don't need to do any of this. Use the tests instead.

Acknowledgements

Development of #14, #16, and #22 was generously sponsored by Transloadit.

License

MIT.

terraform-inventory's People

Contributors

adammck avatar alexandrevoilab avatar anarcher avatar andidog avatar andrefbsantos avatar atadilo avatar atikhono avatar b-m-f avatar ballad89 avatar botto avatar cgroschupp avatar coryjamesfisher avatar djm avatar dmarby avatar fstern avatar generalconsensus avatar jamiefdhurst avatar jjungnickel avatar jonasi avatar jonnymcc avatar jsecchiero avatar kruftik avatar kvz avatar larstobi avatar lazartravica avatar markopolo123 avatar mattpep avatar mbainter avatar quatre avatar schrej avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-inventory's Issues

OpenStack support?

Hi Adam,

Nice job here.
Will you support Openstack provider anytime soon? I'd appreciate it a lot :)

Thanks in advance.

Regards,
Adrián.

Remove Terraform dependency

Now that the tfstate is plain JSON, we can read that directly. No need to include hashicorp/terraform (which is rather difficult to build, since it has external dependencies) to decode the binary format. This will make it possible to install this tool with go get, and will make writing a brew formula much easier.

Outputs in modules not handled properly, clobber global outputs

Terraform output values from modules are not made available to Ansible,
and in fact sub-module outputs clobber the root module's.

Test case setup

.
├── main.tf
├── mymodule
│   ├── main.tf
│   └── vars.tf -> ../vars.tf
├── terraform.tfstate
└── vars.tf

main.tf

provider "aws" {
  region = "${var.aws_region}"
}

resource "aws_instance" "rootinstance" {
  instance_type = "t2.micro"
  ami           = "${var.myami}"
  subnet_id     = "${var.subnet_id}"
  key_name      = "${var.key_name}"

  tags {
    Name = "test-rootinstance"
  }
}

output "testoutput" {
  value = "global output"
}

module "module1" {
  source = "./mymodule"
  name   = "test-module1"
}

module "module2" {
  source = "./mymodule"
  name   = "test-module2"
}

mymodule/main.tf

resource "aws_instance" "moduleinstance" {
  instance_type = "t2.micro"
  ami           = "${var.myami}"
  subnet_id     = "${var.subnet_id}"
  key_name      = "${var.key_name}"

  tags {
    Name = "${var.name}"
  }
}

variable "name" {}

output "testoutput" {
  value = "mymodule name=${var.name}"
}

Results

Terraform

$ terraform apply
(...)
module.module1.aws_instance.moduleinstance: Creation complete

Apply complete! Resources: 3 added, 0 changed, 3 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

Outputs:

testoutput = global output

Outputs:

$ terraform output
testoutput = global output
$ terraform output -module=module1
testoutput = mymodule name=test-module1
$ terraform output -module=module2
testoutput = mymodule name=test-module2

List mode invocation:

Note that vars.testouput === "mymodule name=test-module2" instead of "global output"

$ terraform-inventory -list|jq .
{
  "%_1": [
    "10.233.20.21",
    "10.233.20.54",
    "10.233.20.34"
  ],
  "all": {
    "hosts": [
      "10.233.20.21",
      "10.233.20.34",
      "10.233.20.54"
    ],
    "vars": {
      "testoutput": "mymodule name=test-module2"
    }
  },
  "moduleinstance": [
    "10.233.20.21",
    "10.233.20.34"
  ],
  "moduleinstance.0": [
    "10.233.20.21",
    "10.233.20.34"
  ],
  "name_test-module1": [
    "10.233.20.21"
  ],
  "name_test-module2": [
    "10.233.20.34"
  ],
  "name_test-rootinstance": [
    "10.233.20.54"
  ],
  "rootinstance": [
    "10.233.20.54"
  ],
  "rootinstance.0": [
    "10.233.20.54"
  ],
  "type_aws_instance": [
    "10.233.20.21",
    "10.233.20.54",
    "10.233.20.34"
  ]
}

Host mode invocation

Output values are not available:

$ terraform-inventory -host=10.233.20.54 | jq . | grep output
$ terraform-inventory -host=10.233.20.34 | jq . | grep output
$

No output with Release 0.6.1 or 0.7pre with Terraform 0.10.8 local state file

Trying to make use of this module but no matter how I try I get 0 hosts in the output.

# With 0.7pre

TF_STATE=terraform.tfstate ./.terraform/plugins/linux_amd64/terraform-in
ventory -list
{"all":{"hosts":null,"vars":{}}}

# With 0.6.1
TF_STATE=/usr/local/terraform/terraform.tfstate ./.terraform/plugins/lin
ux_amd64/terraform-inventory -list
{}

How to specify the remote user and private key file to be used when using terraform-inventory

Something I can't figure out though is how to define the remote user and path to the private key when using the inventory generated by this tool to connect with ec2 instances. Normally you could do something like this in the inventory from what I've been reading:

[openvpn]
52.54.106.87 ansible_ssh_user=openvpnas ansible_ssh_private_key_file=~/aws-private.pem

However with the dynamic inventory this isn't possible, or at least clear to me how to change my implementation.
I have a bunch of instances in the infrastructure template from terraform, so this is just an example.

currently I can run a test playbook like this, but its not ideal

ansible-playbook -i /usr/local/bin/terraform-inventory --private-key ~/my_key_pair.pem -l openvpn ansible/openvpn.yaml

I thought a suitable workaround would be to use ssh-add, but for some reason, I can't get that command to work with my vagrant ubuntu/xenial64 vb image.

vagrant@openfirehawkserver:/vagrant$ ssh-add ~/my_key_pair.pem 
Could not open a connection to your authentication agent.

Bug: Running --inventory with array variables fails.

Context: running terraform-inventory --inventory on a .tfstate that has array outputs fails with a panic.

In particular this line: https://github.com/adammck/terraform-inventory/blob/master/cli.go#L77 assumes the variable is a string, but in reality it can be an array of strings. I cannot find any substantial documentation on how ansible handles array variables within the inventory file.

This is the closest I've come to: http://stackoverflow.com/questions/18572092/how-specify-a-list-value-as-variable-in-ansible-inventory-file

no output using libvirt

Installed from master today with 'go get -u github.com/adammck/terraform-inventory'

Command works on the fixtures/example.tfstate but fails on a libvirt tfstate file, yielding only:

$ terraform-inventory --list terraform.tfstate {"all":{"hosts":[],"vars":{}}}

The terraform.tfstate file in question is attached.
terraform.tfstate.txt

Inventory spot instances?

Would it be possible for terraform-inventory to include spot instances in the inventory? Specifically, when a spot request is fulfilled, is there any way to discover the IP of the resulting instance?

Any plans for support for Digitalocean?

At the moment AWS is hard-wired in. An example digitalocean droplet terraform state file the values are slightly different (in terms of IP address etc..) e.g. -

"digitalocean_droplet.test": {
    "type": "digitalocean_droplet",
    "primary": {
        "id": "4597742",
        "attributes": {
            "id": "4597742",
            "image": "11137507",
            "ipv4_address": "46.111.41.144",
            "locked": "false",
            "name": "test-0",
            "region": "lon1",
            "size": "512mb",
            "ssh_keys.#": "1",
            "ssh_keys.0": "54:0r:91:05:db:b7:7f:47:ba:3b:w2:7e:8u:dc:5d:45",
            "status": "active"
        }
    }
}

create new release

Dear @adammck ,

Version 0.6 does not feature the required change to get the correct ip address or GCE. Can you create a new release with the lastest changes?

Thanks!

ssh auth

How do you specify which ssh user and key to use?

Cannot get --list to work

I'm struggling here. I'm not sure if I'm doing something wrong, but I can't get the command to work at all.
I'm using this with terraform 0.12.0

aenglema@ubuntu_template:~/terraform$ terraform-inventory --list
Usage: terraform-inventory [options] path
path: this is either a path to a state file or a folder from which `terraform commands` are valid

My terraform.tfstate file is in the same directory. Terraform show -json works.

Windows Support

The v0.5 release includes Windows builds, but to be honest I have no idea whether they work, because I don't have any Windows machines to test them on. It would be greatly appreciated if someone on Windows could try it out and post the results here.

hostvars aren't being generated with hostname as keys

Currently when I use terraform-inventory I get hostvars with keys by ip address. this is a problem because it means I cannot query hostvars by hostname, and ip adresses are not known for me to query. I would normally hope to be able to use hostvars to be able to query an ip address by host name.

Is it possible to change this behaviour to be able to define the keys better in the inventory/hostvars? normally in an inventory if the ip adress was prepended with the host name it would solve it I think.

ok: [localhost] => {
"hostvars": {
"13.210.124.113": {
"ami": "ami-05fdd828e5a7530b0",
"ansible_check_mode": false,
"ansible_diff_mode": false,
"ansible_facts": {},
"ansible_forks": 5,
"ansible_host": "13.210.241.143",
"ansible_inventory_sources": [
"/usr/local/bin/terraform-inventory"
],
"ansible_playbook_python": "/usr/bin/python",
"ansible_run_tags": [
"all"
],
"ansible_skip_tags": [],
"ansible_ssh_private_key_file": "~/my_key_pair.pem",
"ansible_verbosity": 0,
"ansible_version": {
"full": "2.7.8",

set ansible_python_interpreter

I'm creating CoreOS cluster through Terraform and to provision this machines with ansible i need to set ansible_python_interpreter in the inventory file. Is there a way to set this variable with terraform-inventory?

Doesn't find any hosts with remote state

The merge of #41 supposedly enables remote state.
However this doesn't actually seems to be true.

I use s3 with the following configuration:

terraform {
    backend "s3" {
        bucket = "some.bucket.name"
        key = "terraform.tfstate"
        region = "us-east-1"
    }
}

My .terraform/terraform.tfstate contains:

{
    "version": 3,
    "serial": 0,
    "lineage": "abcdef01-2345-6789-abcd-ef0123456789",
    "backend": {
        "type": "s3",
        "config": {
            "bucket": "some.bucket.name",
            "key": "terraform.tfstate",
            "region": "us-east-1"
        },
        "hash": 01234567890123456789
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

Yet terraform-inventory -list results in: {"all":{"hosts":null,"vars":{}}}

Bug: Running terraform inventory on empty tfstate returns invalid inventory

Context

I define a cluster-name for some infrastructure I'm provisioning in aws. Before I run terraform apply I want to run terraform destroy on that cluster-name to make sure that any previous/existing infrastructure is cleaned up before I create new infrastructure.

Bug

If I pull tfstate for cluster-name that does not exist and then try to run an ansible playbook I get the following error:

ERROR! The file /usr/local/bin/terraform-inventory is marked as executable, but failed to execute correctly. If this is not supposed to be an executable script, correct this with `chmod -x /usr/local/bin/terraform-inventory`.
You defined a group "all" with bad data for the host list:
 {u'hosts': None, u'vars': {<vars here>}}
'utf8' codec can't decode byte 0x80 in position 24: invalid start byte
'utf8' codec can't decode byte 0x80 in position 24: invalid start byte

If I run

/usr/local/bin/terraform-inventory --list <empty-tfstate> | jq .

I get

{
  "all": {
    "hosts": null,
    "vars": {
      <vars here>
    }
  }
}

If I write this to a file and point ansible to it directly I get:

ERROR! 'NoneType' object is not iterable
./test-inventory:2: Expected key=value host variable assignment, got: {

If I change the null field to an empty json array as below:

{
  "all": {
    "hosts": [], 
    "vars": {
      <vars here>
    }
  }
}

And then run ansible with this file it runs as expected.

Ansible HostGroup Support

At present there doesn't appear to be a way to output the data from terraform as Ansible Host Groups to allow for multiple hosts to be configured via a single play.

For example, a play that starts with "hosts: webservers" doesn't know that "aws_instance.webserver.0" should belong to the "webservers" ansible group.

It would be nice to have some way of mapping aws_instance.webserver.* to a group or similar.

Does it support AWS EIP?

Currently, it returns public IP only when we have associate_public_ip_address = true no matter if EIP is attached or not.
And if we do not associate public IP, it returns the private IP.

How can we make it return EIP of the AWS instance?

Use private IP, not public IP?

Is it possible to extract the private IP, instead of public IP? I'm using TF + TF-Inventory, but accessing the servers via a bastion host. It's been a very useful tool so far, thanks!

Support for TF_KEY_NAME

Hi,

I the TF_KEY_NAME override in the source, which is very useful because some servers are only accessable on their private_ip.
Should TF_KEY_NAME be documented? And will it stay supported?
Thanks

Oracle Public Cloud support

All,

I am currently working on Oracle Public cloud infrastructure provisioning using Terraform. can I use the this for syncing inventory with tf state file. does it work?

windows host and ansible tower on linux

we had ansible tower installed on rhel. Does terraform and ansible should be on same machine for this plugin to work?? I am connecting windows is that any different in installing???

How to compile?

I cloned the repo, but there's no make file. I tried the go get github.com/adammck/terraform-inventory command - it ran with exit code 0 but I don't have an executable, it doesn't appear the go get actually did anything.

Not compatible with Terraform v0.12 and the new state pattern

Hi there,

the Terraform v0.12 update broke terraform-inventory. It can no longer read or interprete the new state file pattern.

Error Message:
Usage: terraform-inventory [options] path
path: this is either a path to a state file or a folder from which terraform commands are valid

With best regards

Expected key=value host variable assignment, got: 3,

Im getting the following error when running against local tfstate file:
$: ansible-playbook --inventory-file=terraform.tfstate deploy/playbook.yml
[WARNING]: * Failed to parse /*
/terraform-mariadb-cluster/terraform.tfstate with ini plugin:
*/terraform-mariadb-
cluster/terraform.tfstate:2: Expected key=value host variable assignment, got:
3,
[WARNING]: Unable to parse *
/terraform-mariadb-cluster/terraform.tfstate as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
ERROR! playbooks must be a list of plays

However when I run the terraform-inventory --list terraform.tfstate I do get the following:
{"%4":["54.82.253.204","54.173.45.87","54.152.14.48"],"db":["54.82.253.204","54.173.45.87","54.152.14.48"],"db.0":["54.82.253.204"],"db.1":["54.173.45.87"],"db.2":["54.152.14.48"],"dc_aws":["54.82.253.204","54.173.45.87","54.152.14.48"],"name_db-01":["54.82.253.204"],"name_db-02":["54.173.45.87"],"name_db-03":["54.152.14.48"],"role_dbhost":["54.82.253.204","54.173.45.87","54.152.14.48"],"sshuser*":["54.82.253.204","54.173.45.87","54.152.14.48"],"type_aws_instance":["54.82.253.204","54.173.45.87","54.152.14.48"]}

Can you explain what might be the issue here?

Ansible tower using terraform inventory?

Hi there,

How can integrate the terraform inventory script with ansible tower? . My plan is to run playbook from tower and i need to sync all my hostname information to ansible inventory. can you please help on this?

Is it possible to combine static and dynamic inventories (or multiple inventories) with terraform-inventory?

Are there suggested best practices for combining multiple inventories with this tool?

I was reading this blog post about how to combine multiple inventories with similar tools to terraform-inventory.
It suggests to point your inventory to a folder, and multiple inventory files will be combined.
I'm not sure if its possible to do it here though, because terraform-inventory is an executable.

http://allandenot.com/devops/2015/01/16/ansible-with-multiple-inventory-files.html

Using group vars with GCE

hey @adammck ,

I would like to use group vars to separate production and staging. Currently the names that are exported are the same for production and staging and since the GCE inventory does not list the tags (the tags are provided using a list and not a hashmap) i cannot differentiate between staging and production at the moment and thus i need two separate playbooks. If the tags would be exported i could differentiate between staging and production in the inventory. If you know of another way please let me know.

Thanks!

Keep getting "Unable to find an inventory file" error with ansible-playbook

Hey there, first of all thanks for the great tool! I'm trying to use this but keep getting:

ERROR: Unable to find an inventory file, specify one with -i ?

I'm in the directory with my tfstate file and running terraform-inventory --list works just fine:

terraform-inventory --list
{"appservers.0":["52.25.148.113"],"appservers.1":["52.33.36.249"],"appservers.2":["52.11.196.90"]}%

If it matters, these are AWS instances and I've created three of them using Terraforms "count" param when defining the resource. I'm using Ansible 1.9.4, Terraform 0.6.8, and presumably the latest of terraform-inventory (followed command in the README, version is reported as "unknown" though).

--inventory-file=terraform-inventory does not yield hosts

Following the README currently does not find the proper executable script for inventory. Instead, I found I had to use which terraform-inventory to reverse the path of the executable. Here's some example output (which has been trimmed):

$ terraform-inventory --list
{"db":["54.208.224.51"], . . . }
$ ansible -i terraform-inventory all -m ping
[WARNING]: provided hosts list is empty, only localhost is available

$ ansible -i `which terraform-inventory` all -m ping
54.208.224.51 | . . .

I'm on Ansible 2.0.0.2 currently.

Mixed order

At the moment the order of the output seems random, not sorted alphabetically or grouped by instance type.

Any way it could be sorted?
Thanks

Add support for reading multiple state files

It isn't uncommon for people to have multiple environments in terraform, but want to present a single inventory to ansible.

It would be useful to be able to read multiple state files and then merge the results into a single inventory.

Any plans to support Microsoft Azure?

Hi there! I like the idea of decoupling Terraform and Ansible in the way suggested by terraform-inventory. Wanna share thoughts on support for Microsoft Azure? Cheers!

Digital Ocean?

Does this work with Digital Ocean? Would love to see that happen. I currently am running some janky shells scripts to get Terraform and Ansible to play nicely.

Use .terraform/terraform.tfstate when local state is unavailable

Description

Feature request.

Currently, terraform inventory checks the current working directory for the terraform.tfstate file. When configured to use a remote state, terraform places the state file in the .terraform/ directory instead.

It would be useful for terraform-inventory to check for .terraform/terraform.tfstate if it doesn't find the state in the root of the terraform configuration. This would make life simpler for folks using remote state, and might help avoid the need to wrap terraform-inventory to pull state from a s3 bucket or other remote location.

Steps to reproduce

  1. Create a S3 bucket for remote state
  2. Configure Terraform to use remote state
  3. Confirm that the state was successfully pushed, and that a copy exists in .terraform
  4. Remove the local state files.

Configuration example

terraform remote config -backend=s3 \
  -backend-config="bucket=myproject-terraform-state" \
  -backend-config="key=myproject/terraform.tfstate" \
  -backend-config="region=us-east-1"

Versions

  • terraform-inventory version '0.6.1'
  • Terraform v0.7.0

Additional information

Remote state include a remote block to preserve the remote configuration. This seems to cause no issues for terraform-inventory. I can confirm that terraform-inventory reads the file just fine.

Spot instance name tags do not appear in inventory list

Description

When listing the inventory, spot instances are listed by terraform resource type and terraform resource name, but not by any of their tags.

Background

Terraform does not manage spot instances directly. Instead, it creates spot requests, which are eventually fulfilled by Amazon. It does however check whether the spot requests are fulfilled, and includes the ip_address, DNS, and resource name of the resulting instance.

Behavior

terraform-inventory does list spot-instances that appear as fulfilled spot-instance-requests the terraform state file. It does not include any of the tags associated with the spot instance request.

Steps to reproduce

Create a spot instance configuration

provider "aws" {
  region = "us-west-2"
}

resource "aws_spot_instance_request" "spot_worker" {
  ami = "ami-8118d4e1"
  associate_public_ip_address = true
  instance_type = "t1.micro"
  key_name = "ctbarbou"
  spot_price = ".10"
  spot_type = "persistent"
  wait_for_fulfillment = true

  tags = {
    Name = "spot"
    Role = "demo"
  }
}

Apply configuration

terraform apply
aws_spot_instance_request.spot_worker: Creating...
  ami:                         "" => "ami-8118d4e1"
  associate_public_ip_address: "" => "true"
  availability_zone:           "" => "<computed>"
  ebs_block_device.#:          "" => "<computed>"
  ephemeral_block_device.#:    "" => "<computed>"
  instance_state:              "" => "<computed>"
  instance_type:               "" => "t1.micro"
  key_name:                    "" => "ctbarbou"
  network_interface_id:        "" => "<computed>"
  placement_group:             "" => "<computed>"
  private_dns:                 "" => "<computed>"
  private_ip:                  "" => "<computed>"
  public_dns:                  "" => "<computed>"
  public_ip:                   "" => "<computed>"
  root_block_device.#:         "" => "<computed>"
  security_groups.#:           "" => "<computed>"
  source_dest_check:           "" => "true"
  spot_bid_status:             "" => "<computed>"
  spot_instance_id:            "" => "<computed>"
  spot_price:                  "" => ".10"
  spot_request_state:          "" => "<computed>"
  spot_type:                   "" => "persistent"
  subnet_id:                   "" => "<computed>"
  tags.%:                      "" => "2"
  tags.Name:                   "" => "spot"
  tags.Role:                   "" => "demo"
  tenancy:                     "" => "<computed>"
  vpc_security_group_ids.#:    "" => "<computed>"
  wait_for_fulfillment:        "" => "true"
aws_spot_instance_request.spot_worker: Still creating... (10s elapsed)
aws_spot_instance_request.spot_worker: Still creating... (20s elapsed)
aws_spot_instance_request.spot_worker: Still creating... (30s elapsed)
aws_spot_instance_request.spot_worker: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

List inventory

${HOME}/go/bin/terraform-inventory --list | python -mjson.tool
{
    "spot_worker": [
        "54.244.41.190"
    ],
    "spot_worker.0": [
        "54.244.41.190"
    ],
    "type_aws_spot_instance_request": [
        "54.244.41.190"
    ]
}

List host

 ${HOME}/go/bin/terraform-inventory --host 54.244.41.190 | python -mjson.tool
{
    "ami": "ami-8118d4e1",
    "associate_public_ip_address": "true",
    "block_duration_minutes": "0",
    "ebs_block_device.#": "0",
    "ephemeral_block_device.#": "0",
    "id": "sir-03ckpwx7",
    "instance_type": "t1.micro",
    "key_name": "ctbarbou",
    "private_dns": "ip-172-31-12-189.us-west-2.compute.internal",
    "private_ip": "172.31.12.189",
    "public_dns": "ec2-54-244-41-190.us-west-2.compute.amazonaws.com",
    "public_ip": "54.244.41.190",
    "root_block_device.#": "0",
    "security_groups.#": "0",
    "source_dest_check": "true",
    "spot_bid_status": "fulfilled",
    "spot_instance_id": "i-8c0a4651",
    "spot_price": ".10",
    "spot_request_state": "active",
    "spot_type": "persistent",
    "tags.%": "2",
    "tags.Name": "spot",
    "tags.Role": "demo",
    "vpc_security_group_ids.#": "0",
    "wait_for_fulfillment": "true"
}

Expected result

Inventory list includes name_spot and role_demo.

Actual result

Inventory list includes instance by resource_name and resource type, but not by tag.

Additional information

The terraform state includes the spot request, which contains information about the resulting instance. It does not actually contain instance data itsself. terraform-inventory seems to add the instance based on this data.

Note that Tags are not inherited from spot-request to spot-instance. E.g.

# Spot request tags
aws ec2 describe-tags --filter="Name=resource-id,Values=sir-03ckpwx7"
{
    "Tags": [
        {
            "ResourceType": "spot-instances-request",
            "ResourceId": "sir-03ckpwx7",
            "Value": "spot",
            "Key": "Name"
        },
        {
            "ResourceType": "spot-instances-request",
            "ResourceId": "sir-03ckpwx7",
            "Value": "demo",
            "Key": "Role"
        }
    ]
}
# Spot instance tags
aws ec2 describe-tags --filter="Name=resource-id,Values=i-8c0a4651"
{
    "Tags": []
}

In this case, the most desirable behavior would be to organize the inventory list based on the spot request tags, rather than attempting to pull the actual spot-instance tags from AWS.

Actually applying tags to a spot-instance is quite complex.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.