Code Monkey home page Code Monkey logo

ansible-modules-hashivault's Introduction

Ansible Modules Hashivault

Ansible modules for Hashicorp Vault.

Latest version Downloads CI License: MIT

Install this Ansible module:

  • via pip:
pip install ansible-modules-hashivault
  • via ansible-galaxy (requires hvac>=0.7.2):
ansible-galaxy install 'git+https://github.com/TerryHowe/ansible-modules-hashivault.git'
Note: The hashicorp lookup plugin does not work with this last install method (ansible/ansible#28770). You can fallback to the build-in lookup plugin: hashi_vault

In most cases the Hashicorp Vault modules should be run on localhost.

Environmental Variables

The following variables need to be exported to the environment where you run ansible in order to authenticate to your HashiCorp Vault instance:

  • VAULT_ADDR: url for vault
  • VAULT_SKIP_VERIFY=true: if set, do not verify presented TLS certificate before communicating with Vault server. Setting this variable is not recommended except during testing
  • VAULT_AUTHTYPE: authentication type to use: token, userpass, github, ldap, approle
  • VAULT_LOGIN_MOUNT_POINT: mount point for login defaults to auth type
  • VAULT_TOKEN: token for vault
  • VAULT_ROLE_ID: (required by approle)
  • VAULT_SECRET_ID: (required by approle)
  • VAULT_USER: username to login to vault
  • VAULT_PASSWORD: password to login to vault
  • VAULT_CLIENT_KEY: path to an unencrypted PEM-encoded private key matching the client certificate
  • VAULT_CLIENT_CERT: path to a PEM-encoded client certificate for TLS authentication to the Vault server
  • VAULT_CACERT: path to a PEM-encoded CA cert file to use to verify the Vault server TLS certificate
  • VAULT_CAPATH: path to a directory of PEM-encoded CA cert files to verify the Vault server TLS certificate
  • VAULT_AWS_HEADER: X-Vault-AWS-IAM-Server-ID Header value to prevent replay attacks
  • VAULT_NAMESPACE: specify the Vault Namespace, if you have one

Documentation

There are a few simple examples in this document, but the full documentation can be found at:

https://terryhowe.github.io/ansible-modules-hashivault/modules/list_of_hashivault_modules.html

Reading and Writing

The following example writes the giant secret with two values and then reads the fie value. The hashivault_secret module is kv2 by default:

---
- hosts: localhost
  tasks:
    - hashivault_secret:
        secret: giant
        data:
            foo: foe
            fie: fum
    - hashivault_read:
        secret: giant
        key: fie
        version: 2
      register: vault_read

The lookup plugin:

- set_fact:
    looky: "{{lookup('hashivault', 'giant', 'foo', version=2)}}"

The hashivault_write, hashivault_read and the lookup plugin assume the /secret mount point. If you are accessing another mount point, use mount_point:

---
- hosts: localhost
  tasks:
    - hashivault_secret_engine:
        name: stories
        backend: generic
    - hashivault_write:
        mount_point: /stories
        secret: stuart
        data:
            last: 'little'
    - hashivault_read:
        mount_point: /stories
        secret: stuart
        key: last
    - set_fact:
        book: "{{lookup('hashivault', 'stuart', 'last', mount_point='/stories')}}"

Version 2 of KV secret engine is also supported, just add version: 2:

---
    - hashivault_read:
        mount_point: /stories
        version: 2
        secret: stuart
        key: last
    - set_fact:
        book: "{{lookup('hashivault', 'stuart', 'last', mount_point='/stories', version=2)}}"

Initialization, Seal, and Unseal

The real strength of this module is all the administrative functions you can do. See the documentation mentioned above for more, but here is a small sample.

You may init the vault:

---
- hosts: localhost
  tasks:
    - hashivault_init:
      register: vault_init

You may also seal and unseal the vault:

---
- hosts: localhost
  vars:
    vault_keys:  "{{ lookup('env','VAULT_KEYS') }}"
  tasks:
    - hashivault_status:
      register: vault_status
    - block:
        - hashivault_seal:
          register: vault_seal
      when: "{{vault_status.status.sealed}} == False"
    - hashivault_unseal:
        keys: '{{vault_keys}}'

Action Plugin

If you are not using the VAULT_ADDR and VAULT_TOKEN environment variables, you may be able to simplify your playbooks with an action plugin. This can be some somewhat similar to this example action plugin.

Developer Note

One of the complicated problems with development and testing of this module is ansible/module_utils/hashivault.py is not a directory in itself which in my opinion is a problem with ansible. Because of this limitation with ansible, pip install -e . does not work like it would for other projects. Two potential ways to work around this issue are either use the link.sh script in the top level directory or run for every change:

rm -rf dist; python setup.py sdist
pip install ./dist/ansible-modules-hashivault-*.tar.gz

License

MIT.

ansible-modules-hashivault's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-modules-hashivault's Issues

hashivault_token_create does not work with authtype: approle

I'm currently testing this hashivault module for python, but I stumbled accross the following error:

I'm trying to generate a token for a defined approle, for this I'm retrieving the role_id and secret_id using the provided methods. According to the documentation the following should then retrieve the token:

---
- name: Test Vault lookup
  hosts: localhost
  no_log: false
  tasks:
  - hashivault_approle_role_id:
      name: ansible
    register: 'vault_role_id'

  - debug:
      msg: "{{ vault_role_id.id }}"

  - hashivault_approle_role_secret_create:
      name: ansible
    register: 'vault_role_secret_create'

  - debug:
      msg: "{{ vault_role_secret_create.secret_id }}"

  - hashivault_token_create:
      display_name: approle
      authtype: approle
      role_id: "{{ vault_role_id.id }}"
      secret_id: "{{ vault_role_secret_create.secret_id }}"
      policies: [ "ansible" ]
    register: vault_token

  - debug:
      msg: "{{ vault_token }}"

  environment:
    VAULT_ADDR: "https://localhost:8200"
    VAULT_CACERT: "cert"
    VAULT_TOKEN: fancy_token

But hashivault_token_create fails with permission denied message.

With the provided vault token, i can create a token for this approle. So policies do work. However when looking through the code and the requests made against vault i saw the following:

Here are the requests against vault when using vault write auth/approle/login role_id=foo secret_id=bar:

{
    "time": "2018-07-04T14:50:58.412152016Z",
    "type": "response",
    "auth": {
      "client_token": "hmac-sha256:d1fa1f74ad5aeb2e198bda60a20c9cc74b7d7fbe9f5631de6e51a2bf82b52942",
      "accessor": "hmac-sha256:0a78fed8478ca10f3591783c8b6c56f3849d7cb77d6b9ed877a14d26fbd613fc",
      "display_name": "approle",
      "policies": [
        "ansible",
        "default"
      ],
      "token_policies": [
        "ansible",
        "default"
      ],
      "metadata": {
        "role_name": "ansible"
      },
      "entity_id": "e275625c-b6af-286d-c093-c752e2e02352"
    },
    "request": {
      "id": "32347833-16b7-dd55-c1b8-a5853b81efc9",
      "operation": "update",
      "client_token": "hmac-sha256:c79ed6c84930065b1b238258f9a1c069e3e22739c09a62a023c585365009381c",
      "client_token_accessor": "hmac-sha256:5dc18a22ce8b289516a722352ec1118cf11a62b59f6a415de3698551713de69e",
      "path": "auth/approle/login",
      "data": {
        "role_id": "hmac-sha256:e35acd0afd583494841c244afda6068c0713405e2d9cb1090dae03966d4c1af1",
        "secret_id": "hmac-sha256:600a3283e135246f80bd0f397d21201069900dee479efc5a699fde521f615328"
      },
      "policy_override": false,
      "remote_address": "ip",
      "wrap_ttl": 0,
      "headers": {}
    }

There is one request and one response, delivering the token.

However when running the playbook above, the hashivault_token_create method returns two requests:

{
    "time": "2018-07-04T14:50:48.351173813Z",
    "type": "response",
    "auth": {
      "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
      "accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
      "display_name": "approle",
      "policies": [
        "ansible",
        "default"
      ],
      "token_policies": [
        "ansible",
        "default"
      ],
      "metadata": {
        "role_name": "ansible"
      },
      "entity_id": "e275625c-b6af-286d-c093-c752e2e02352"
    },
    "request": {
      "id": "55c81279-89c5-8a50-4aaf-5b42015ebefe",
      "operation": "update",
      "client_token": "",
      "client_token_accessor": "",
      "path": "auth/approle/login",
      "data": {
        "role_id": "hmac-sha256:e35acd0afd583494841c244afda6068c0713405e2d9cb1090dae03966d4c1af1",
        "secret_id": "hmac-sha256:600a3283e135246f80bd0f397d21201069900dee479efc5a699fde521f615328"
      },
      "policy_override": false,
      "remote_address": "ip",
      "wrap_ttl": 0,
      "headers": {}
    },
    "response": {
      "auth": {
        "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
        "accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
        "display_name": "approle",
        "policies": [
          "ansible",
          "default"
        ],
        "token_policies": [
          "ansible",
          "default"
        ],
        "metadata": {
          "role_name": "ansible"
        },
        "entity_id": ""
      }
    },
    "error": ""
  },

The first one looks fine, it's also running against auth/approle/login, however afterwards the second requests comes in:

{
    "time": "2018-07-04T14:50:48.353911896Z",
    "type": "request",
    "auth": {
      "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
      "accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
      "display_name": "approle",
      "policies": [
        "ansible",
        "default"
      ],
      "token_policies": [
        "ansible",
        "default"
      ],
      "metadata": {
        "role_name": "ansible"
      },
      "entity_id": "e275625c-b6af-286d-c093-c752e2e02352"
    },
    "request": {
      "id": "af03ed86-b7a9-02ea-aae6-746e36b1ca18",
      "operation": "update",
      "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
      "client_token_accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
      "path": "auth/token/create",
      "data": {
        "display_name": "hmac-sha256:670f1c96d3608d1341311565d8cfd382b434dc426901f565fa5b96f852625705",
        "explicit_max_ttl": null,
        "id": null,
        "meta": null,
        "no_default_policy": false,
        "no_parent": false,
        "num_uses": null,
        "policies": [
          "hmac-sha256:609017dcd80855ab6fb4f80f6302cc1a495a11b50244db39ee24add07ebcaff3"
        ],
        "renewable": null,
        "ttl": null
      },
      "policy_override": false,
      "remote_address": "ip",
      "wrap_ttl": 0,
      "headers": {}
    },
    "error": "permission denied"
  },
  {
    "time": "2018-07-04T14:50:48.35415999Z",
    "type": "response",
    "auth": {
      "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
      "accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
      "display_name": "approle",
      "policies": [
        "ansible",
        "default"
      ],
      "token_policies": [
        "ansible",
        "default"
      ],
      "metadata": {
        "role_name": "ansible"
      },
      "entity_id": "e275625c-b6af-286d-c093-c752e2e02352"
    },
    "request": {
      "id": "af03ed86-b7a9-02ea-aae6-746e36b1ca18",
      "operation": "update",
      "client_token": "hmac-sha256:dce4762e4cd4c2f2cbd78c091e611704bb66c4d785d1b3963bbf9b717555ad54",
      "client_token_accessor": "hmac-sha256:b1174011ce92a280463679888f6d78107b48e7e31b0f2e684f779e796d24916e",
      "path": "auth/token/create",
      "data": {
        "display_name": "hmac-sha256:670f1c96d3608d1341311565d8cfd382b434dc426901f565fa5b96f852625705",
        "explicit_max_ttl": null,
        "id": null,
        "meta": null,
        "no_default_policy": false,
        "no_parent": false,
        "num_uses": null,
        "policies": [
          "hmac-sha256:609017dcd80855ab6fb4f80f6302cc1a495a11b50244db39ee24add07ebcaff3"
        ],
        "renewable": null,
        "ttl": null
      },
      "policy_override": false,
      "remote_address": "ip",
      "wrap_ttl": 0,
      "headers": {}
    },
    "response": {
      "data": {
        "error": "hmac-sha256:f1307ff81a70a896c1383b979abb0b49d692a1c113d8b451a3507dccf45f11cb"
      }
    },
    "error": "1 error occurred:\n\n* permission denied"

The second request is running against auth/token/create, which fails, as my policy does not allow this. I think the problem is related to the call of create_token here as this will be executed after the AppRoleClient call.

I guess this bug applies to the other authtypes as well, as the create_token function is called afterwards again and the hashivault_auth method is already requesting the tokens.

Read a Vault token from $HOME/.vault-token

If you run vault auth, it will write its token (on Linux and MacOS at least) to $HOME/.vault-token. It would be helpful if this module understood that file.

As a workaround, before running Ansible, I currently need to

export VAULT_TOKEN=$(cat ~/.vault-token)

I think at https://github.com/TerryHowe/ansible-modules-hashivault/blob/master/ansible/module_utils/hashivault.py#L14 a more involved default token check would work

def get_default_token():
  """Get a default Vault token from an environment variable or a file."""
  if 'VAULT_TOKEN' in os.environ:
    return os.environ['VAULT_TOKEN']
  token_file = os.path.expanduser('~/.vault-token')
  if os.path.exists(token_file):
    with open(token_file, 'r') as f:
      return f.read()
  return ''

token = dict(required=False, default=get_default_token(), type='str', no_log=True)

I'm not 100% clear on the environment where this can run (e.g., will Ansible run this code on the target system?) or where the token lives on Windows hosts.

enabling the kubernetes backend in 0.8.3

I'm a bit confused as to how to use the hashivault_write command to configure this backend.
Here are the instructions on how to setup the backend: https://github.com/aws-samples/aws-workshop-for-kubernetes/tree/master/04-path-security-and-networking/401-configmaps-and-secrets#configure-kubernetes-auth-backend

Specifically need to configure it by writing a token, the certificate authority and the server api to this backend.

 vault write auth/kubernetes/config \
  token_reviewer_jwt=$REVIEWER_TOKEN  \
  kubernetes_host=<api-server> \
  kubernetes_ca_cert=@~/.kube/kops.crt

thoughts?

Unable to import VAULT_ADDR

Hi I have configured vault server and try to connect it is taking value as 127.0.0.1.

Here is the ansible log
FAILED! => {"failed": true, "msg": "{{ lookup('hashivault', 'dev-env', 'central_user')}}: Error reading vault dev-env/central_user: Exception: HTTPSConnectionPool(host='127.0.0.1', port=8200): Max retries exceeded with url: /v1/secret/dev-env (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f59f5d83c10>: Failed to establish a new connection: [Errno 111] Connection refused',))\n"}

Environment variable confusion

Hi, Sorry if my question is kind of immature but I really stumbled accross the following error:

fatal: [etc01]: FAILED! => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "authtype": "token", 
            "ca_cert": "", 
            "ca_path": "", 
            "client_cert": "", 
            "client_key": "", 
            "data": {
                "certificates": "abc"
            }, 
            "password": "", 
            "role_id": "", 
            "secret": "kafka", 
            "secret_id": "", 
            "token": "e75b7556-762d-ed89-1dca-9c3923187257", 
            "update": false, 
            "url": "", 
            "username": "", 
            "verify": true
        }
    }, 
    "msg": "Exception: Invalid URL '/v1/secret/kafka': No schema supplied. Perhaps you meant http:///v1/secret/kafka?", 
    "rc": 1
}

I'm trying to run playbook on remote machine. All neccessary variable are set but still no go. Looks like module is not reading env. The token is readed form .vault-token on remote machine instead from env.
This is how I run it:

ansible-playbook -i poc.inv -e VAULT_ADDR=https://bunsen:8200 -e VAULT_TOKEN=e75b7556-762d-ed89-1dca-9c3923187258 poc.yml

poc.yml

---
- hosts: poc
  roles:
    - vault

main.yml:

---
- name: Write secrets
  hashivault_write:
    secret: 'kafka'
    data:
      certificates: 'abc'
  register: 'vault_write'

Any thoughts about how it should be done, please?
Cheers,
BB

The module does not work with ansible install via apt

Hi,
Module does not work on servers or ansible was installed directly via apt and not pip
OS: Ubuntu 16.04 x64 update to date

root@srv01:~# ansible --version
ansible 2.4.3.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

Exemple

apt update
apt install software-properties-common
apt-add-repository ppa:ansible/ansible
apt update

apt install ansible
pip install ansible-modules-hashivault

export VAULT_ADDR="https://vault.mydomain:8200"
export VAULT_SKIP_VERIFY=1

Exemple: test.yml

- hosts: localhost
  tasks:
  - name: "blabla"
    hashivault_status:
    register: 'vault_status'
  - assert: { that: "{{vault_status.changed}} == False" }
  - assert: { that: "{{vault_status.status.progress}} == 0" }
  - assert: { that: "{{vault_status.status.sealed}} == False" }
  - assert: { that: "{{vault_status.rc}} == 0" }
root@srv01:~# ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/root/test.yml': line 3, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
  - name: "blabla"
    ^ here


The error appears to have been in '/root/test.yml': line 3, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
  - name: "blabla"
    ^ here

exception type: <class 'ansible.errors.AnsibleParserError'>
exception: no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/root/test.yml': line 3, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
  - name: "blabla"
    ^ here

The same playbook on a server with Ansible install via pip

root@srv02:~# ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] ************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [blabla] ***************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [assert] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [assert] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [assert] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [assert] ***************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY RECAP ******************************************************************************************************************************************************************************************************************************
localhost                  : ok=6    changed=0    unreachable=0    failed=0

Broken Compatibility for write for pki

We were using your module to issue pki certs from vault, now we do not get the data back sent by vault only a message written by your module... it would be useful to actually have it working again

hashivault_write always tries to read before writing, even when update=false

Hey,

As the title says. I can see this was changed recently but it actually breaks a number of things.

One example is the aws secret backend, which is configured at the path aws/config/root. But this path is write-only, so trying to read first throws an error.

Likewise for approles, you can generate a secret-id by writing to auth/approle/role/my-role/secret-id, but attempting to read from this path is an error.

It also fails if you simply don't have permission to read the endpoint you're trying to write to (entirely possible for automated scripts)

Can you revert the change please? Or find a way of doing the same that doesn't error in these cases?

Broken tests

Hi,

I started using this module, and I noticed that the tests you have don't seem to be functioning properly. Using the run.sh script to execute tests (and providing a value for VAULT_ADDR).

+ ansible-playbook -v test_init.yml
No config file found; using defaults
 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available


PLAY [localhost] ***************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [hashivault_init] *********************************************************
changed: [localhost] => {"changed": true, "keys": ["44f7a5f432cc9748b6825ac7d9bf83dad26b5bc5beceeed0a7b84fc8e4f9c079bb", "6398c40062bfd88d9ede0aeef186f5aa3562d1b71bae6ff766d735205446670382", "fcd62dfae4ecfc1d5a4bfd35d10143419a8298029586a0f15119e8dbdd5ef99424", "f9262f525b4f6135dd9ad4abbee3961296d2ca3cebadab2f103d5d12c6039931ad", "ea30c58398bdf1ee5be2bc51bef0c1c429f05e16adf9159cc1d2b45d0a14ef6f13"], "keys_base64": ["RPel9DLMl0i2glrH2b+D2tJrW8W+zu7Qp7hPyOT5wHm7", "Y5jEAGK/2I2e3gru8Yb1qjVi0bcbrm/3Ztc1IFRGZwOC", "/NYt+uTs/B1aS/010QFDQZqCmAKVhqDxURno291e+ZQk", "+SYvUltPYTXdmtSrvuOWEpbSyjzrrasvED1dEsYDmTGt", "6jDFg5i98e5b4rxRvvDBxCnwXhat+RWcwdK0XQoU728T"], "rc": 0, "root_token": "55407d6d-a6f8-7370-41a2-dd62e9af83c2"}

TASK [assert] ******************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional check failed", "skipped": true}

TASK [assert] ******************************************************************
fatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check ''keys' in {u'keys': [u'44f7a5f432cc9748b6825ac7d9bf83dad26b5bc5beceeed0a7b84fc8e4f9c079bb', u'6398c40062bfd88d9ede0aeef186f5aa3562d1b71bae6ff766d735205446670382', u'fcd62dfae4ecfc1d5a4bfd35d10143419a8298029586a0f15119e8dbdd5ef99424', u'f9262f525b4f6135dd9ad4abbee3961296d2ca3cebadab2f103d5d12c6039931ad', u'ea30c58398bdf1ee5be2bc51bef0c1c429f05e16adf9159cc1d2b45d0a14ef6f13'], u'keys_base64': [u'RPel9DLMl0i2glrH2b+D2tJrW8W+zu7Qp7hPyOT5wHm7', u'Y5jEAGK/2I2e3gru8Yb1qjVi0bcbrm/3Ztc1IFRGZwOC', u'/NYt+uTs/B1aS/010QFDQZqCmAKVhqDxURno291e+ZQk', u'+SYvUltPYTXdmtSrvuOWEpbSyjzrrasvED1dEsYDmTGt', u'6jDFg5i98e5b4rxRvvDBxCnwXhat+RWcwdK0XQoU728T'], u'changed': True, u'root_token': u'55407d6d-a6f8-7370-41a2-dd62e9af83c2', u'rc': 0}' failed. The error was: template error while templating string: expected token ':', got 'string'. String: {% if 'keys' in {u'keys': [u'44f7a5f432cc9748b6825ac7d9bf83dad26b5bc5beceeed0a7b84fc8e4f9c079bb', u'6398c40062bfd88d9ede0aeef186f5aa3562d1b71bae6ff766d735205446670382', u'fcd62dfae4ecfc1d5a4bfd35d10143419a8298029586a0f15119e8dbdd5ef99424', u'f9262f525b4f6135dd9ad4abbee3961296d2ca3cebadab2f103d5d12c6039931ad', u'ea30c58398bdf1ee5be2bc51bef0c1c429f05e16adf9159cc1d2b45d0a14ef6f13'], u'keys_base64': [u'RPel9DLMl0i2glrH2b+D2tJrW8W+zu7Qp7hPyOT5wHm7', u'Y5jEAGK/2I2e3gru8Yb1qjVi0bcbrm/3Ztc1IFRGZwOC', u'/NYt+uTs/B1aS/010QFDQZqCmAKVhqDxURno291e+ZQk', u'+SYvUltPYTXdmtSrvuOWEpbSyjzrrasvED1dEsYDmTGt', u'6jDFg5i98e5b4rxRvvDBxCnwXhat+RWcwdK0XQoU728T'], u'changed': True, u'root_token': u'55407d6d-a6f8-7370-41a2-dd62e9af83c2', u'rc': 0} %} True {% else %} False {% endif %}"}
	to retry, use: --limit @/Users/telant/src/ansible-modules-hashivault
/functional/test_init.retry

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=1

It appears that in test_init, you're looking for the existence of keys and root_token, but the condition in your assert isn't evaluating properly, and quickly errors out.


My system details:

$ python
Python 2.7.13 (default, Feb  6 2017, 17:59:14)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2, ansible
>>> jinja2.__version__
'2.9.3'
>>> ansible.__version__
'2.2.0.0'

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.12
BuildVersion:	16A323

I have some potential plans to extend the functionality of what you've started, but I need to get these tests working first.

I was wondering if you were interested in me sending a pull request once I have the tests working again? (And perhaps also when I have my extended functionality?)

Thanks!

Vault Secret Engine 2

I can do the following using the Vault CLI client:

$ vault kv list secret/certificates
Keys
----
cert1.domain.com
cert2.domain.com
cert3.domain.com

My Secret Engine config looks like so:

image

With some tasks like so in play:

tasks:
    - hashivault_list:
        secret: 'secret/certificates'
      register: cert_list
      delegate_to: localhost
    - debug:
        var: cert_list

I would expect to have a list, however, it's empty:

TASK [debug] ********************************************************************************************************************************************************************************************************************************************************
ok: [nsdnginx1] => {
    "cert_list": {
        "changed": false, 
        "failed": false, 
        "rc": 0, 
        "secrets": []
    }
}

Feature Request: Enable and Configure Secret Backends

There seems to be an API for enabling and configuring auth and audit backends, but not for secret backends.

Is this a feature that is planned for ansible-modules-hashivault? This would help complete the automation of Vault setup.

hashivault_write with update tries to write empty string to unset values

Maybe this is not a problem in most cases, but for some secrets it matters.
E.g. here:

- hashivault_write:
    update: true
    secret: "/auth/aws/role/myrole"
    data:
      auth_type: ec2
      policies:
      - mypolicy
      disallow_reauthentication: true
      bound_iam_instance_profile_arn: "arn:aws:iam::1234:instance-profile/myinstanceprofile"

This works if the secret is new but if it already exists and I've changed the bound_iam_instance_profile_arn value then it fails with

"Exception: failed updating the unique ID of ARN \"\": &errors.errorString{s:\"unrecognized arn: contains 1 colon-separated parts, expected 6\"}"

My understanding is that this is because it tries to write empty string to the other arn-field (bound_iam_principal_arn)

.vault_token is read with the newline

When using the module with the vault_token, it reads the file with the \n and fails
One can strip that from the file, but the module should be flexible and ignore any newline, as vaut do. so this simple fix solve this issue:

diff --git a/ansible/module_utils/hashivault.py b/ansible/module_utils/hashivault.py
index b548457..dc2d14a 100644
--- a/ansible/module_utils/hashivault.py
+++ b/ansible/module_utils/hashivault.py
@@ -88,5 +88,5 @@ def hashivault_default_token():
     token_file = os.path.expanduser('~/.vault-token')
     if os.path.exists(token_file):
         with open(token_file, 'r') as f:
-            return f.read()
+            return f.read().rstrip()
     return ''

hashivault_secret_enable to enable with type

Hi,

I am trying to use hashivault_secret_enable to enable ssh-backend, something similar to below. And I don't see any equivalent options in the py. Do you know if there is a quick way to do this perhaps!

vault secrets enable -path=ssh-prod ssh 

Cheers, Raj

When behind nginx proxy, vault https url is wrongly constructed

We have vault configured at /vault/ uri behind nginx proxy. So all our vault urls are https://localhost/vault/v1... likewise. Ngnix is configured to forward the request to http://localhost:8200/.

However, while using following module the resulting url is constructed as https://localhost/v1/sys/init and failing with 404 error. Ideally it should be https://localhost/vault/v1/sys/init. This is due to urljoin function in /usr/lib/python2.7/site-packages/hvac/v1/init.py

hashivault_init:
url: 'https://localhost/vault/'
secret_shares: 1
secret_threshold: 1
verify: False
register: init_status

Error in module import

ansible --version
ansible 2.1.1.0

Playbook

- hosts: qa
  vars:
    url: 'http://localhost'
  tasks:
    - hashivault_status:
      register: 'vault_status'
    - debug: msg="{{ vault_status }}"

Error message that i get:
{"failed": true, "msg": "Could not find imported module support code for hashivault_status. Looked for either * or hashivault"}

hashivault_write reports changes in "update" mode when passing strings for int values

Hey,

There are some places in the Vault API where you can write a string value, but reading returns an int value. For example, the max_ttl value inside a role in the SSH backend. Though the documentation example (https://www.vaultproject.io/api/secret/ssh/index.html#read-role) shows it returning a string, it actually returns a plain integer value:

curl -H "X-Vault-Token: ${VAULT_TOKEN}" ${VAULT_ADDR}/v1/ssh/roles/testrole
{"request_id":"ed7b1f29-5f64-0016-9c09-3f214876b26c","lease_id":"","renewable":false,"lease_duration":0,"data":{"allow_bare_domains":false,"allow_host_certificates":false,"allow_subdomains":false,"allow_user_certificates":true,"allow_user_key_ids":false,"allowed_critical_options":"","allowed_domains":"","allowed_extensions":"","allowed_users":"*","default_critical_options":{},"default_extensions":{"permit-pty":""},"default_user":"centos","key_id_format":"","key_type":"ca","max_ttl":0,"ttl":1800},"wrap_info":null,"warnings":null,"auth":null}

The way the hashivault_write module checks for changes means that if you write a string value from Ansible, it will always be reported as a change, even though nothing has actually changed.

The obvious workaround is to set an integer value in Ansible instead, but this isn't always possible - for example, when the value is templated (as in our case) it will always be a string, and so will always be marked as changed.

Would it be possible to have the check relaxed slightly, so that the string value is not treated as a change?

It's worth mentioning that there are other cases in which you get a similar "always changed" behaviour due to reads not matching the writes, e.g. when using values like "30m" for a TTL value, which the read returns as "1800". I doubt that's so easily fixable though, as it depends on what field is being written/read, and there are inconsistencies between the backends (as some return "1800" and some return "30m0s")

Need v2 K/V secrets engine support

Server started with:

% vault server -dev

Example code from https://pypi.org/project/ansible-modules-hashivault/:

% export VAULT_ADDR='http://127.0.0.1:8200'
% cat >example.yml <<EOF
---
- hosts: localhost
  vars:
    foo_value: 'fum'
    fie_value: 'fum'
  tasks:
    - hashivault_status:
      register: 'vault_status'
    - hashivault_write:
        secret: 'giant'
        data:
            foo: '{{foo_value}}'
            fie: '{{fie_value}}'
      register: 'vault_write'
    - hashivault_read:
        secret: 'giant'
        key: 'fie'
      register: 'vault_read'
EOF
% ansible-playbook example.yml
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [hashivault_status] **********************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [hashivault_write] ***********************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "rc": 1
}

MSG:

Exception: {"request_id":"4c06a0be-9e8d-a4f0-6603-1c96d258a876","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":null,"warnings":["Invalid path for a versioned K/V secrets engine. See the API docs for the appropriate API endpoints to use. If using the Vault CLI, use 'vault kv put' for this operation."],"auth":null}

        to retry, use: --limit @/some/path/example.retry

PLAY RECAP ************************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=1   
%

Manual query:

% vault kv put secret/foo value=bar
Key              Value
---              -----
created_time     2018-05-18T15:25:50.377136792Z
deletion_time    n/a
destroyed        false
version          1
% vault kv get secret/foo       
====== Metadata ======
Key              Value
---              -----
created_time     2018-05-18T15:25:50.377136792Z
deletion_time    n/a
destroyed        false
version          1

==== Data ====
Key      Value
---      -----
value    bar
% vault write secret/bar value=baz 
Error writing data to secret/bar: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/secret/bar
Code: 404. Errors:


WARNING! The following warnings were returned from Vault:

  * Invalid path for a versioned K/V secrets engine. See the API docs for the
  appropriate API endpoints to use. If using the Vault CLI, use 'vault kv put'
  for this operation.
%

Versions:

% vault --version
Vault v0.10.1 ('756fdc4587350daf1c65b93647b2cc31a6f119cd+CHANGES')
% ansible --version
ansible 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/foo/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.5 (default, May 11 2018, 04:00:52) [GCC 8.1.0]
% uname -a
Linux something 4.16.8-1-ARCH #1 SMP PREEMPT Wed May 9 11:25:02 UTC 2018 x86_64 GNU/Linux
%

documenting enviromental variables

I had hard time finding which environmental variables this module uses.
After looking at the source code for a while, I found what I was looking for, but it would have been easier if it was more readily available.

Vault seal operation requesting client token

Hello,

There's a logical error when calling the hashivault_seal module that's making it throw

Exception: 1 error occurred:\n\n* missing client token

Vault doesn't require a client token to seal the vault, just the VAULT_TOKEN (aka root token) is required.

Here's my vault status:

vagrant@slave:~$ vault status
Sealed: false
Key Shares: 6
Key Threshold: 4
Unseal Progress: 0
Unseal Nonce:
Version: 0.7.3
Cluster Name: vault-cluster-93661e49
Cluster ID: 93684931-7133-0fd5-ccab-d46dee4fefd4

High-Availability Enabled: false

Sample play:

- name: seal the Vault
  hashivault_seal:
    url: "http://0.0.0.0:8200"
  register: vault_seal

Play output

TASK [debian-hashivault : seal the Vault] *******************************************************************************
fatal: [slave-1]: FAILED! => {"changed": false, "msg": "Exception: 1 error occurred:\n\n* missing client token", "rc": 1}

PLAY RECAP **************************************************************************************************************
slave-1                    : ok=2    changed=0    unreachable=0    failed=1

Feature Request: Check Mode Support

This feature request is for the module to support check_mode as provided by Ansible. This will assist operators to confirm what changes will be made to their secrets prior to pushing them out.

Change log?

Do you publish a changelog for this nice module somewhere else? it is not easy to keep up with the changes if one had to look at the commits

Use Variables in Key when Writing Secrets

What:
It appears that when writing a secret, the key you are writing cannot be or contain an ansible variable.

Example:

  • name: Store some password in HVAULT
    hashivault_write:
    url: "{{HVAULT_URL}}"
    verify: false
    authtype: userpass
    username: ansible
    password: "{{HVAULT_PASS}}"
    secret: '{{Group|upper}}/{{Env_Tier|upper}}'
    data:
    "someuser{{ User_Suffix }}": "{{ someuser_Password }}"
    delegate_to: localhost

Expected Results:

under-
data:
"someuser{{ User_Suffix }}": "{{ someuser_Password }}"

The module should be able to append whatever the value of {{User_Suffix}} is to the key it writes in Hashicorp Vault. So if the value of {{User_Suffix}} is P the module should write the secret in Vault as someuserP.

Actual Results
In testing, the module is writing the literal string someuser{{ User_Suffix }} as the KEY in Vault, even though the value is correctly written in Vault as the variable value for {{ someuser_Password }}

hashivault_init does not return the generated keys in result

- hashivault_init:
  register: 'vault_init'
  environment:
    VAULT_ADDR: "{{ VAULT_ADDR }}"
  when: vault_init
  run_once: yes

- debug: msg="{{vault_init}}"

The debug returns:

TASK [platform-vault-init : debug] ******************************************************************
ok: [10.0.20.143] => {
    "msg": true
}

It's missing the keys.

This seems to be the culprit: https://github.com/TerryHowe/ansible-modules-hashivault/blob/master/ansible/modules/hashivault/hashivault_init.py#L101

Any ideas?

dependecy to module hvac - ansible-modules-hashivault is not self contained

Hi,

when executing any vault task with this ansible module I get an ERROR:

fatal: [10.248.71.17]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Shared connection to 10.248.71.17 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File "/tmp/ansible_Zojwta/ansible_module_hashivault_init.py", line 75, in \r\n from ansible.module_utils.hashivault import *\r\n File "/tmp/ansible_Zojwta/ansible_modlib.zip/ansible/module_utils/hashivault.py", line 3, in \r\nImportError: No module named hvac\r\n", "msg": "MODULE FAILURE", "rc": 0}

meaning on the target host I have to install the hvac python module which is not easily possible, because i have no internet connection on this machine where Vault is installed. So I can't use pip or any other package manager to install it over ansible.

This module should be standalone runnable (self contained). Please include all your dependent python modules in your artifact.

thx
Andi

Using hashivault_write to configure certs

Hi,
I am trying to add a cert policy like below:

- name: Configure cert backend with cert & policy
  hashivault_write:
    url: "{{ vault_addr }}"
    verify: no
    token: "{{ vault_token }}"
    secret: "/auth/cert/certs/intermediate_ca"
    data:
      display_name: intermediate_ca 
      policies: test
      certificate: \@/vault/etc/vault-intermediate-ca.cert.pem

The error I am running into is "fatal: [10.0.38.225]: FAILED! => {"changed": false, "failed": true, "msg": "Exception: failed to parse certificate", "rc": 1}". Am I missing something?

There are several other configs using hashivault module that is going fine.

If I execute the following directly, it looks like working. So I am pretty sure that the cert itself is proper and the permission, etc are valid.

vault write auth/cert/certs/intermediate_ca
display_name=intermediate_ca
policies=test
certificate=@/vault/etc/vault-intermediate-ca.cert.pem

I am using ansible-playbook v2.2.1.0

Additional info (with -vvv):

fatal: []: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"authtype": "token",
"data": {
"certificate": "\@/vault/etc/vault-intermediate-ca.cert.pem",
"display_name": "intermediate_ca",
"policies": "test"
},
"password": null,
"secret": "/auth/cert/certs/intermediate_ca",
"token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"update": false,
"url": "https://vault.andromeda.cisco.local:8200",
"username": null,
"verify": false
},
"module_name": "hashivault_write"
},
"msg": "Exception: failed to parse certificate",
"rc": 1
}

New release

Hi,

I would like to use the contribution added by this commit: f51143c
But there is no release on pip with this commit.

@TerryHowe do you have an ETA for the release generation ?

Confused on correct approach to use this plugin

Hi,
Thank you for writing this wonderful plugin. Using a simple playbook I can read values out of my vault instance no problem. My playbook looks like this:

#
# This playbook is making the assumption that VAULT_ADDR is exported in the environment.
#
- hosts: localhost
  gather_facts: false
  connection: local

  tasks:
    #    - debug: msg="The  jhaals vault plugin username secret is:{{ lookup('vault', 'secret/internal/CI/test', 'username') }}"

  - name: "Retrieve secrets from vault using the hashi_vault plugin"
    hashivault_read:
      secret: '/secret/internal/CI/test'
      key: 'username'
    register: 'vault_read_result'

  - debug:
      msg: "username={{ vault_read_result.value }}"

So in another playbook, the one that I want to retreive the actual vault key value, delegating to localhost, I get an error and I believe that it means that I've messed up somewhere because the module doesn't know the url to get to the vault nor does it know that I'm logged into my vault instance.
Here is the snippet of my playbook that uses the hashivault module:



- name: "Get the enterprise drone license from hashicorp vault"
  delegate_to: localhost
  hashivault_read:
    secret: 'secret/internal/CI/ccs_foo_com/jenkins-drone/drone-license'
    key: 'DRONE_LICENSE'
  register: 'vault_read_result'
  tags:
    - drone-server
    - drone-read

- name: "Setting fact DRONE_LICENSE"
  delegate_to: localhost
  set_fact:
    DRONE_LICENSE: "{{vault_read_result.value}}"
  when: "{{ vault_read_result.rc }} == 0"
  tags:
    - drone-server
    - drone-read

- assert: { that: "{{vault_read_result.rc}} == 0" }

Obviously I want to have my localhost machine get access to vault and then set the key value to a fact to be inserted into a template later in the playbook. The error I'm getting is below:

fatal: [ci-master.ops.tropo.com -> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "authtype": "token",
            "ca_cert": "",
            "ca_path": "",
            "client_cert": "",
            "client_key": "",
            "key": "DRONE_LICENSE",
            "password": "",
            "register": null,
            "role_id": "",
            "secret": "secret/internal/CI/ccs_foo_com/jenkins-drone/drone-license",
            "secret_id": "",
            "token": "",
            "url": "",
            "username": "",
            "verify": true
        }
    },
    "msg": "Exception: Invalid URL '/v1/secret/secret/internal/CI/ccs_foo_com/jenkins-drone/drone-license': No schema supplied. Perhaps you meant http:///v1/secret/secret/internal/CI/ccs_foo_com/jenkins-drone/drone-license?",
    "rc": 1
}

I've seen some mention of your action plugins, however I'm surprised that this work flow doesn't work. Am I using this method wrong? Should I run all my queries for secrets from localhost, set the facts then proceed to do the rest of the playbook and not do any type of delegation?
thanks,
Matt

policy creation is not idempotent

My playbook is simple:

  1. cat the contents of a *.hcl policy file and register to {{ file_contents }}
  2. create the policy using "hashivault_policy_set"
    • name is derived from the policy file name (same name but minus the .hcl)
    • content is derived from the {{ file_contents.stdout }}

error

successful run

$ ansible-playbook tasks/set_policy/vault_set_policy.yml -i tasks/set_policy/inv -e "policy_file=./policies/app_awx.hcl"

PLAY [all] **********************************

TASK [cat file] ****************************
changed: [localhost -> localhost]

TASK [set policy from file contents] **
changed: [localhost -> localhost]

PLAY RECAP ****************************************
localhost : ok=2 changed=2 unreachable=0 failed=0

rerunning yields failure

$ ansible-playbook tasks/set_policy/vault_set_policy.yml -i tasks/set_policy/inv -e "policy_file=./policies/app_awx.hcl"

PLAY [all] *****************************************

TASK [cat file] ********************************************
changed: [localhost -> localhost]

TASK [set policy from file contents] ****************************************************
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "msg": "Exception: u'rules'", "rc": 1}

PLAY RECAP *********************************************
localhost : ok=1 changed=1 unreachable=0 failed=1

playbook

---

- hosts: all
  strategy: linear
  gather_facts: false
  tasks:

  - name: cat file
    local_action: shell cat "{{ policy_file }}"
    register: file_contents

  - name: set policy from file contents
    hashivault_policy_set:
      name: "{{ (policy_file.split('.')[0]) | basename | string }}"
      rules: "{{ file_contents.stdout }}"
    delegate_to: localhost

verbose error

fatal: [localhost -> localhost]: FAILED! => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "authtype": "token", 
            "ca_cert": "", 
            "ca_path": "", 
            "client_cert": "", 
            "client_key": "", 
            "name": "app_awx", 
            "password": "", 
            "role_id": "", 
            "rules": "path \"secret/*\" { capabilities = [\"list\"] }\n\npath \"secret/data/lob/*\" { capabilities = [\"list\", \"read\", \"update\", \"create\", \"delete\"] }\n\npath \"secret/metadata/lob/*\" { capabilities = [\"list\", \"read\", \"delete\"] }", 
            "secret_id": "", 
            "token": "<my token>", 
            "url": "https://vault.example.com:8200", 
            "username": "", 
            "verify": false
        }
    }, 
    "msg": "Exception: u'rules'", 
    "rc": 1
}

hashivault_approle_role_secret_create does not support wrap_ttl

Hello,

It seems that this module hashivault_approle_role_secret_create does not support wrap_ttl:

  hashivault_approle_role_secret_create:
    token: "{{ vault_admin_token.stdout }}"
    name: goldfish
    wrap_ttl: "5m"
  register: 'vault_role_secret_create'

Thank you in advance for your help.

Regards,

Khang

Installation guide

Hi.

It will be good if you add installation guide of your modules to ansible. Because it's not clear for not Python developers and for people who hasn't enough experience in Ops to do that.

Thank you.

approle support

Hi,

This is quite a neat package, however I am struggling to get this working with approle. Is there a way to get this working so that we can read or write secrets just using app role - i.e role_id and secret_id set, and not generate a token and set that token as an environment variable instead?

regards, Raju

Module fails to unseal

  • hosts: localhost
    vars:
    vault_keys: "{{ lookup('env','VAULT_KEY1', 'VAULT_KEY2', 'VAULT_KEY3') }}"
    tasks:
    • name: print environment
      debug:
      msg: "key vaule{{item}}"
      with_items:

      • "{{ vault_keys }}"
    • name: Unseal status
      hashivault_status:
      register: 'vault_status'

    • name: Print the status of vault
      debug:
      msg: "vault environment sealed ? {{ vault_status.status.sealed }}"

    • name: Unseal Vault if Sealed
      hashivault_unseal:
      keys: '{{vault_keys}}'
      when: vault_status.status.sealed == True

I had to set vault keys separately on bash, and even then it fails to unseal with below error

TASK [print environment] ********************************************************************************************************************************************************************************************************************
Friday 30 March 2018 11:34:54 +0100 (0:00:00.539) 0:00:00.618 **********
ok: [localhost] => (item=RExlyUib/KhCRCHHWpLc0+CGFlrrk2gFM0FaKc4C7IrH,uIFpZScsRF74+SISdPe+icR1g1yP3P2p9z/Nh/fP9Mkm,KHw8ph30CmNF5Bh0AyBiVIZ6M8lVXKzPkFLnOUbS8wVQ) => {
"item": "RExlyUib/KhCRCHHWpLc0+CGFlrrk2gFM0FaKc4C7IrH,uIFpZScsRF74+SISdPe+icR1g1yP3P2p9z/Nh/fP9Mkm,KHw8ph30CmNF5Bh0AyBiVIZ6M8lVXKzPkFLnOUbS8wVQ",
"msg": "key vauleRExlyUib/KhCRCHHWpLc0+CGFlrrk2gFM0FaKc4C7IrH,uIFpZScsRF74+SISdPe+icR1g1yP3P2p9z/Nh/fP9Mkm,KHw8ph30CmNF5Bh0AyBiVIZ6M8lVXKzPkFLnOUbS8wVQ"
}

TASK [Unseal status] ************************************************************************************************************************************************************************************************************************
Friday 30 March 2018 11:34:54 +0100 (0:00:00.028) 0:00:00.647 **********
ok: [localhost]

TASK [Print the status of vault] ************************************************************************************************************************************************************************************************************
Friday 30 March 2018 11:34:54 +0100 (0:00:00.420) 0:00:01.067 **********
ok: [localhost] => {
"msg": "vault environment sealed ? True"
}

TASK [Unseal Vault if Sealed] ***************************************************************************************************************************************************************************************************************
Friday 30 March 2018 11:34:54 +0100 (0:00:00.035) 0:00:01.103 **********
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Exception: 'key' must be a valid hex or base64 string", "rc": 1}
[WARNING]: Failure using method (v2_runner_on_failed) in callback plugin (</usr/lib/python2.7/site-packages/ansible/plugins/callback/mail.CallbackModule object at 0x1927ed0>): 'module_name'

Question: why does the lookup module convert the result value to string?

A quick question about the behavior of the lookup module:

I've observed that the data received from Vault is always converted to a string, regardless of its real type.

For example, if I have a list stored in Vault:

Key            Value
---            -----
my_secrets     [my_secret1 my_secret2 my_secret3]

then the value returned by the lookup plugin is a string instead of a proper list, i.e.:

"[u'my_secret1', u'my_secret2', u'my_secret3']"

If I assign this result directly to an Ansible variable, it gets automatically converted to a list by the Ansible's templating engine. It is not possible, however, to use the result directly in a more complex Jinja2 expression, such as:

"{{ lookup('hashivault', '/secret/example', 'my_secrets') | first }}"

The above expression does not return the first element from the list (as I would expect), but the first character of the string ([).

Is there any reason why the lookup plugin cannot return a proper list in such a case?

hashivault_read_to_file does not mesh well with bastion hosts and firewalls

Much to my surprise, the module requires that the read operation takes place on the local ansible host. First of all, that adds a python dependency on the Ansible host that I wasn't planning on. Secondly, it requires that I can route from my Ansible host to the Vault host. In my case, that's not allowed.

I would prefer if the action was run on the target node that wants to run hashivault_read_to_file. Can you explain why this task needs to be local and if we could make it configurable?

lookup plugin not functioning

Learning to use this great plugin. Not sure if im using it wrong or if there is a bug. Notice, im able to read the value when i use hashivault_read but not when i use the lookup('hashivault') method

ansible --version: 2.5.2
vault version: 0.10.1

Playbook:

tasks:

## USE hashivault_read module ## WORKS!
  - name: read value
    hashivault_read:
      secret: test
      key: password
    register: mypass

  - name: output pass
    debug:
      var: mypass

## USE lookup plugin ## DOES NOT WORK
  - name: look up value
    debug:
      msg: "{{ lookup('hashivault', 'test', 'password') }}"

Output:

TASK [read value] **************************
ok: [localhost]

TASK [output pass] *************************
ok: [localhost] =>
  mypass:
    changed: false
    failed: false
    rc: 0
    value: P@ssw0rd

TASK [look up value] ***********************
fatal: [localhost]: FAILED! =>
  msg: |-
    An unhandled exception occurred while running the lookup plugin 'hashivault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Error reading vault test/password: Exception: <HTML><HEAD>
                    <TITLE>Network Error</TITLE>
                    </HEAD>
                    <BODY>
                    <FONT face="Helvetica">
                    <big><strong></strong></big><br />
                    </FONT>
                    <blockquote>
                    <TABLE border=0 cellPadding=1 width="80%">
                    <TR><TD>
                    <FONT face="Helvetica">
                    <big>Network Error (dns_server_failure)</big>
                    <br />
                    <br />
                    </FONT>
                    </TD></TR>
                    <TR><TD>
                    <FONT face="Helvetica">
                    Please confirm you typed your URL correctly.<P> Your request could not be processed because an error occurred contacting the DNS server.
                    </FONT>
                    </TD></TR>
                    <TR><TD>
                    <FONT face="Helvetica">
                    The DNS server may be temporarily unavailable, or there could be a network problem.
                    </FONT>
                    </TD></TR>
                    <TR><TD>
                    <FONT face="Helvetica" SIZE=2>
                    <br />
                    For assistance, contact your network support team.
                    </FONT>
                    </TD></TR>
                    </TABLE>
                    </blockquote>
                    </FONT>

Also, just a comment on the documentation: looky: "{{lookup('hashivault', 'giant', 'foo')}}"

is looky a module you provide? If not, it was confusing to me and you may want to update to use a debug example instead.

How do I enable github auth backend and assign policies to teams/users?

Hello, first of all I want to say thanks for the great module :)

But I need some help here... I stuck on getting github auth configured.

I basically need few things:

  1. vault write auth/github/config organization=my-awesome-org
  2. vault write auth/github/map/teams/default value=very-cool-policy-for-default-team-in-my-awesome-org
  3. vault write auth/github/map/users/developer value=very-cool-policy-for-user-developer-in-my-awesome-org

it basically should be covered by hashivault_write / hashivault_read as there is nothing special, but I do not see this covered in the docs so wonder if this is supported...

Allow use of hashivault without requiring root

I'm have been trying trying to use hashivault as much as standalone as possible, ie: not requiring root to install hashivault and be able to run directly everything after a git clone... but i'm failing hard, ansible do not even search for the module in other places

i did open a bug in ansible:
ansible/ansible#46004

but they report it's not possible right now...

Have you any workaround for this?

init returns a dictionary with a key "keys" which is python internal method

hashivault_init retuns a result that looks like this

ok: [host] => { "msg": { "changed": true, "keys": [ "x", "x", "x", "x", "x" ], "keys_base64": [ "x", "x", "x", "x", "x" ], "rc": 0, "root_token": "x" } }

Now when I try to store these keys to a safe storage (credstash in my case) I am iterating over this structure

- name: store unseal keys to credstash command: credstash put unseal-{{ item.0 }} {{ item.1 }} with_items: "{{ init_result.keys }}"

I get an error as keys is a python internal method.

For keys_base64 everything works as expected.

Would you consider renaming "keys" to something that would not cause this issue?

Lookup plugin doesn't work with environment vars in tasks.

When i set VAULT_ADDR directly in tasks with environment parameter, hashivault lookup plugin doesn't load vars.

Exemple :

    - name: USER Create User
      tags: account_create_user
      environment:
        VAULT_ADDR: "{{ vault.url }}"
        VAULT_TOKEN: "{{ account_vault.root_token }}"
        VAULT_SKIP_VERIFY: true    
      vars:
        vault_path: "/test/{{ item }}"
        vault_username: "{{ lookup('hashivault', vault_path, 'username') }}"
        vault_password: "{{ lookup('hashivault', vault_path, 'password') }}"
        vault_state:    "{{ lookup('hashivault', vault_path, 'state') }}"
        vault_key:      "{{ lookup('hashivault', vault_path, 'key') }}"
        account_home:   "/home/{{ vault_username }}"          
      with_items: "{{ account_group_fusion }}"
      user:
        name:            "{{ vault_username }}"
        comment:         "{{ account_description }}"
        shell:           "{{ account_shell }}"
        password:        "{{ vault_password|password_hash('sha512') }}"
        groups:          "{{ account_groups }}"
        createhome:      "{{ account_createhome }}"
        home:            "{{ account_home }}"
        state:           "{{ vault_state }}"
        update_password: "{{ account_reset_password }}"

Output :

An unhandled exception occurred while templating '{{ lookup('hashivault', vault_path, 'username') }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'hashivault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Error reading vault /test/[email protected]/username: Exception: HTTPSConnectionPool(host='127.0.0.1', port=8200): Max retries exceeded with url: /v1/test/[email protected] (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fad10ef4b50>: Failed to establish a new connection: [Errno 111] Connection refused',))

PS : vault.url is not https://127.0.0.1:8200

Self-Signed Vault Cert: hashivault_read successful, lookup fails

Versions:
ansible: 2.5.4
ansible-modules-hashivault: 3.9.5
hvac: 0.6.0

Environment variables set:
VAULT_ADDR=https://my.vault.install:8200/
VAULT_CACERT=/Users/atrepca/.certs/vault-ca.pem

This works fine:

---
- hosts: localhost
  gather_facts: False
  - name: Read Vault secret - foo
    hashivault_read:
      secret: 'secret/foo'
      key: 'bar'
    register: 'bar'

This fails:

---
- hosts: localhost
  gather_facts: False
  vars:
    bar: "{{ lookup('hashivault', 'secret/foo', 'bar') }}"
  tasks:
    - debug:
        msg: "bar={{ bar }}"

With a CERTIFICATE_VERIFY_FAILED error:

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ lookup('hashivault', 'secret/foo', 'bar') }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'hashivault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Error reading vault/secret/foo/bar: Exception: HTTPSConnectionPool(host='my.vault.install', port=8200): Max retries exceeded with url: /v1/secret/foo (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),))\n"}

If I export VAULT_SKIP_VERIFY=true, the lookup works as expected.

Module Fails with Self-Signed Vault Certificate

So it appears that this module fails when you have a Vault deployment that is using a self-signed certificate/TLS
TASK [Read values] *********************************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Exception: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)", "rc": 1}

Is there a workaround possible or perhaps the ability to validate_certs: false like other ansible modules?

It seems the only way around this is by setting VAULT_SKIP_VERIFY; it would be nice to be able to set this in the playbook itself.

Key in write data does not expand

Sample:

- hosts: localhost
  vars: 
    - key: "key123"
    - value: "value123"
  tasks:
    - hashivault_write:
      secret: "/cesta/k/jebnutemu/secretu"
        data:
          "{{ key }}": "{{ value }}"
$ vault read /cesta/k/jebnutemu/secretu
Key                    Value
---                    -----
refresh_interval       168h0m0s
{{ key }}              value123

hashivault_approle_role_create does not support secret_id_ttl, period, token_ttl, token_max_ttl

Hello,

Eveything is in the title, for now i'm using a workaround by using the generic end point:

hashivault_write:
    token: "{{ vault_admin_token.stdout }}"
    secret: '/auth/approle/role/goldfish'
    data:
      role_name: "goldfish"
      policies: "default,goldfish"
      secret_id_num_uses: 1
      secret_id_ttl: "5m"
      period: "24h"
      token_ttl: 0
      token_max_ttl: 0

Thank you in advance for your help.

Regards,

Khang

Error while run playbook

I have setup virtualenv
virtualenv -q venv
and install ansible and ansible-modules-hashivault with pip

source venv/bin/activate
pip install -r reqs.txt

After that I run your demo playbook but it fails

- hosts: localhost
  tasks:
    - hashivault_write:
        secret: giant
        data:
            foo: foe
            fie: fum
    - hashivault_read: secret='giant' key='fie' register='fie'
    - debug: msg="Value is {{fie.value}}"

error is:

(venv) ➜  vault-demo git:(master) ✗ ansible-playbook playbook.yml
 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/Users/ivan.skiridomov/PycharmProjects/vault-demo/playbook.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - hashivault_write:
      ^ here


The error appears to have been in '/Users/ivan.skiridomov/PycharmProjects/vault-demo/playbook.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
    - hashivault_write:
      ^ here

Functional test lookup plugin broken on mac

TASK [Test lookup plugin] ******************************************************
ERROR! A worker was found in a dead state

Definitely related to the gather_facts setting

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.