synchronize can't find ssh key when path is relative about ansible-modules-core HOT 23 CLOSED

@Woodham: Your approach doesn't work if ansible_ssh_private_key_file was never specified. For example, when private_key_file is globally set on ansible.cfg.

My fix for this would be setting the private-key with Jinja template. Since the private-key is templatized, you can specify your private-key as:

private_key_file = {{inventory_dir}}/../blah.pem

from ansible-modules-core.

Hm, I use the {{inventory_dir}}. Maybe {{playbook_dir}} is not available in the hosts-file.

Here the relevant part of my hosts-file:

[server-group]
server.my-domain.com
[server-group:vars]
ansible_ssh_private_key_file={{inventory_dir}}/key.pem

from ansible-modules-core.

Quick question -

Where was your SSH key relative to your playbook and the directory where you executed ansible?

Also since you mentioned rsync, are you using the synchronize module? Can we see the task line you've used?

Thanks!

from ansible-modules-core.

Sure thing!

Here's the task:

- name: make sure code is on the box and in sync
  synchronize:
    src: ../../../../../
    dest: "{{base_path}}"
    rsync_opts:
      - --include=.bin/***
      - --include=api/***
      - --include=config/***
      - --include=nest/***
      - --include=package.json
      - --include=index.js
      - --exclude='.*'
      - --exclude='*'

Here's a subset of my directory structure:

├── Vagrantfile
├── config
│   ├── db.js
│   └── secrets
│       ├── admiralty.pem
│       ├── github.json
│       ├── nest.pem (this is the key)
│       └── s3.json
├── deploy
│   └── ansible
│       ├── deploy.yml
│       ├── group_vars
│       │   ├── all.yml
│       │   ├── production.yml (path to key defined as a variabe here)
│       │   ├── staging.yml
│       │   └── vagrant.yml
│       ├── handlers
│       │   └── handlers.yml
│       ├── hosts
│       ├── provision.yml (the playbook is here)
│       ├── reset-staging.yml
│       ├── roles
│       │   └── startup
│       │       └── tasks
│       │           └── main.yml (the task is here)
│       └── vagrant.yml
├── index.js
└── package.json

from ansible-modules-core.

I think this has something to do with how the path to the private key is passed to rsync. I believe if Ansible resolved the absolute path before providing it, it would work fine.

from ansible-modules-core.

Just noting that my workaround is now broken in ansible 1.8.2

from ansible-modules-core.

ansible 1.7 produced this, which worked:
rsync --delay-updates -FF --compress --archive --rsh 'ssh -i /path/to/key.pem -o StrictHostKeyChecking=no

ansible 1.8 does this, which doesn't:
rsync --delay-updates -FF --compress --archive --rsh 'ssh -i /path/to/key.pem -S none -o StrictHostKeyChecking=no

from ansible-modules-core.

Figured it out, sort of. The synchronize module doesn't pass -S none to rsync's --rsh option if it is the first command in a playbook.

from ansible-modules-core.

I am experiencing this problem in ansible 1.8.4 along Vagrant.

The same occurs if the private key is referred inside the inventory with 'ansible_ssh_private_key_file':
ansible_ssh_private_key_file=../keys/vagrant-insecure-private-key

Or else, expecting it would automatically solve the path (relative or not - which is something extremely reasonable), it does not.

Without Vagrant, there is the same problem, even when running the "ping" module.
The point is that ansible do not properly handles relative paths within this context.

from ansible-modules-core.

I have been struggling with this issue also and I believe I've discovered a workaround. If you add this task before using synchronise:

    - name: Set  correct ssh key path
      set_fact: 
        ansible_ssh_private_key_file: "{{ ansible_ssh_private_key_file | realpath }}"

Then rsync will use the correct ssh key path :)

from ansible-modules-core.

@kokon Using the following when: on the task should solve that one.

when: ansible_ssh_private_key_file is defined

from ansible-modules-core.

Hi!

Thanks very much for your submission to Ansible. It sincerely means a lot to us.

Testing this on the version you have specified, we cannot reproduce this problem.

Our reasons and evidence for this are:

• I have tested every released version of ansible that contained the synchronize module and with ANSIBLE_PRIVATE_KEY_FILE set, the relative path was always sent as an ssh parameter to rsync and always succeeded to find the file. The relative path was always relative to the playbook directory,

And we suggest that:

• Test with a more recent version of ansible or use any of the workarounds mentioned in this isue. If you continue to see the problem, please open a new issue with exact details of the full ansible-playbook command used, the directory structure, the relative paths and their absolute versions, and which directory you are running from.

As such, we are most likely going to close this ticket. Should you disagree with this conclusion, you may also wish to join our mailing lists, which are very active:

• https://groups.google.com/forum/#!forum/ansible-project - for user questions, tips, and tricks
• https://groups.google.com/forum/#!forum/ansible-devel - for strategy, future planning, and questions about writing code

Thank you once again for this and your interest in Ansible!

from ansible-modules-core.

This was working fine on 2.1.1.0 and previous but on 2.2.0.0 happens again.

    - name: Set  correct ssh key path
      set_fact: 
        ansible_ssh_private_key_file: "{{ ansible_ssh_private_key_file | realpath }}"

Fixes the issue.

My dir structure:

project/
  ansible/
    playbook.yml
    keys/key.pem
  ... project files

From project root I call ansible-playbook ansible/playbook.yml and path to my key.pem is defined in inventory:

... ansible_ssh_private_key_file=ansible/keys/key.pem

It's all fine if having same behaviour in each version, madness is when you change from 1.7 to 2.1 and things work fine but then you update to 2.2 and you still face bugs from 1.7!

from ansible-modules-core.

I've just encountered this issue as well. The problem is that the ssh uses the path to keyfile as relative from inventory directory, but the rsync takes it as relative to playbook directory. So if you want to have your inventory separated from the playbook, you're screwed and you have to use these "hopethisworks" workarounds. I guess that more consistent behaviour with respect to the paths would be very welcome.

from ansible-modules-core.

My recommendation that if you are shipping an ssh_config, that you should reference the path to that ssh_config relative to variables that ansible provides, such as:

• playbook_dir
• inventory_dir
• role_path

However, you will be unable to define ssh_args in your ansible.cfg and instead will have to define it via group_vars or similar, so that you can use runtime variables to impact the path, such as:

ansible_ssh_extra_args: -F {{ inventory_dir }}/ssh_config

from ansible-modules-core.

I've tried to do this:

• put my host in a new group ssh-conf
• created group_vars/ssh-conf under inventory directory with following content:

ansible_ssh_private_key_file: "{{ playbook_dir }}/../vagrant/.vagrant/machines/mymachine/virtualbox/private_key"

This works quite fine, now I can call my playbook and inventory from any path and it always finds the key. However, the synchronize task still fails, because it does not expand the ansible_ssh_private_key_file variable:

fatal: [mymachine]: FAILED! => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --compress --delete-after --recursive --times --rsh 'ssh -i {{ playbook_dir }}/../vagrant/.vagrant/machines/mymachine/virtualbox/private_key -S none -o StrictHostKeyChecking=no -o Port=22222' --rsync-path=\"sudo rsync\" --omit-dir-times --exclude supervisor_config_gener.xml --exclude nemea_status.conf --out-format='<<CHANGED>>%i %n%L' \"/home/me/ansible/inventory/host_files/mymachine/nemea/\" \"vagrant@localhost:/etc/nemea/\"", "failed": true, "msg": "Warning: Identity file {{ not accessible: No such file or directory.\nssh: Could not resolve hostname playbook_dir: Name or service not known\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(235) [sender=3.1.1]\n", "rc": 255}

Why is it not expanded here? I have to use the set_fact workaround in some setup task to properly expand this variable. All of this seems really clumsy when all I want to do is to set ansible_ssh_private_key_file in an inventory file relative to either playbook or inventory file.

from ansible-modules-core.

@thorgrin Does the workaround in #18 (comment) not still work?

    - name: Set  correct ssh key path
      set_fact: 
        ansible_ssh_private_key_file: "{{ ansible_ssh_private_key_file | realpath }}"
      when: ansible_ssh_private_key_file is defined

from ansible-modules-core.

@PriceChild It works for me (ansible 2.1.1.0), but it is a workaround nonetheless. I'd really like to see this relative path thing to work consistently for all tasks.

from ansible-modules-core.

@thorgrin Great, thanks! We removed synchronise from our playbooks around 2.0 due to issues with become but are looking into returning to it because of ansible/ansible#21513 Good to hear it's still an option!

I can't help but think it should be reasonably easy to fix it properly though... maybe I'll give it a go and submit a PR.

from ansible-modules-core.

Just ran into this issue. Fortunately I found a workaround for me, which I'd like to share with you:

My .pem-file resides in the same directory as my hosts-file. With ansible_ssh_private_key_file={{inventory_dir}}/key.pem as a group-variable in the host-file synchronize finds the key and everything seems to work.

It's still a workaround, but imho more lightweight than adding an extra task to the playbook as suggested above.

from ansible-modules-core.

@christopherfrieler Can you post an example of the hosts-fie? This does not work for me, the variable is not substituted in the synchronize command, I stil get:
--rsh 'ssh -i {{ playbook_dir }}/../vagrant/.vagrant/machines/staas-vagrant/virtualbox/private_key

from ansible-modules-core.

I've tried that, but with no luck. I'm running 2.1.1.0.

from ansible-modules-core.

This should be fixed now by ansible/ansible#22133

from ansible-modules-core.