Unable to use docker_service because ansible-docker installs docker-py about ansible-docker HOT 11 CLOSED

Hi,

This role installs Docker using Docker's official Debian / Ubuntu package. Compose is also copied over directly as a binary (not installed through Python).

This role shouldn't be installing docker-py. I would test it on a fresh system.

from ansible-docker.

Hi Nick, more info below!

This role installs Docker using Docker's official Debian / Ubuntu package. Compose is also copied over directly as a binary (not installed through Python)

Indeed I'm talking about Python modules here, not the Go binaries. docker_service uses the docker-compose Python module which as I understand it is an alternative, pure Python implementation of docker-compose : https://pypi.org/project/docker-compose/

This role shouldn't be installing docker-py

I use ansible-docker to login to Docker registries and this what seems to trigger the installation of docker-py:
https://github.com/nickjj/ansible-docker/blob/master/tasks/main.yml#L114

from ansible-docker.

Oh yeah, that's because the Ansible docker_login module requires the docker-py package to be installed. Its dependencies are listed at https://docs.ansible.com/ansible/2.5/modules/docker_login_module.html.

I'm having trouble understanding the real issue though.

• The login module installs docker-py
• The service module requires docker-compose
• This role install docker-py (if you're logging into a registry)
• This role installs Docker Compose

All of the requirements to run both the login and server modules should be there?

To my understanding docker-py is nothing more than Python code to allow for interacting with Docker through Python instead of having to shell out to Docker binary commands. It's not a competing service or daemon. It's just code sitting on your file system until you decide to use it in a Python script.

from ansible-docker.

Sorry for my unclear explanations. I really appreciate your time trying to understand the issue. I just realized that the binary installed by ansible-docker is the same Python implementation which just happen to be packaged as a standalone binary. For some reason I assumed docker-compose was written in Go like Docker itself, which is why I thought it was a separate implementation. I amended my comment above.

The problem is that the docker_service module needs access to Docker Compose Python API: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/docker/docker_service.py#L456
When Docker Compose is installed as binary, the compose Python module isn't available in site-packages and cannot be imported from Python code. This is why I install the docker-compose myself with pip. When I install docker-compose with pip, it installs the docker Python module which conflicts with docker-py.

I've tried to amend ansible-docker so that it installs docker instead of docker-py and it seems to be working for me so far.

Maybe I should create a minimal example playbook that triggers the issue so you can reproduce it?

from ansible-docker.

When you say this:

"I've tried to amend ansible-docker so that it installs docker instead of docker-py and it seems to be working for me so far."

Are you talking about pip installing docker instead of docker-py to meet the docker_login module's requirements? When you do that, both docker_login and docker_service works?

I never installed Compose with pip, mainly because I don't use docker_service or any Ansible Docker modules (except for logging into a registry).

The registry management could be done without docker_login but it's kind of annoying to make it idempotent.

I don't mind switching to using docker instead of docker-py as long as docker_login still works. If you want to set up a test playbook where you use both login and service and it demonstrates it working, that would be a good start.

from ansible-docker.

Are you talking about pip installing docker instead of docker-py to meet the docker_login module's requirements? When you do that, both docker_login and docker_service works?

Yes this is what I meant!

I don't mind switching to using docker instead of docker-py as long as docker_login still works

Yes in my experience, it is the case. In fact, until now I have been using docker_login directly while having installed docker-compose with pip (which installed the docker Python module).

Now that I found out the docker__registries option of ansible-docker, I thought it would be good to switch to it as it makes things slightly neater.

I'll look into making a playbook demonstrating the use case.

from ansible-docker.

Here is the playbook

---

- hosts: all
  become: true
  tasks:
    - name: Install required packages
      apt:
        state: present
        update_cache: true
        name:
          - python-pip
          - python-setuptools

- hosts: all
  become: true
  vars:
    docker__registries:
      - registry_url: registry.gitlab.com
        unsername: gitlab.com
        password: "{{ password }}"
  roles:
    - role: nickjj.docker
  tasks:
    - name: Install docker-compose python module
      pip:
        name: docker-compose
    - name: Copy docker-compose.yml
      copy:
        src: docker-compose.yml
        dest: docker-compose.yml
    - name: Start nginx
      docker_service:
        state: present
        project_src: .

And the docker-compose.yml:

version: "3"

services:
  nginx:
    image: nginx:1.13.6-alpine
    ports:
      - "80:80"

This gives me this error:

TASK [Start nginx] ********************************************************************
fatal: [[email protected]]: FAILED! => {"changed": false, "msg": "Cannot have
both the docker-py and docker python modules installed together as they use the
same namespace and cause a corrupt installation. Please uninstall both packages,
and re-install only the docker-py or docker python module. It is recommended to
install the docker module if no support for Python 2.6 is required. Please note that
simply uninstalling one of the modules can leave the other module in a broken state."}
        to retry, use: --limit @/home/al/essai/ansible/playbook.retry

Then I change nickjj.docker/tasks/main.yml so that it installs docker instead of docker-py:

- name: Install docker for managing Docker login credentials
  pip:
    name: "docker"
  when: docker__registries

Then I reset my VM and run ansible-playbook again and everything works fine. Docker login succeeds and nginx works.

from ansible-docker.

Great, thanks. I will test this in a day or 2 and get back to you.

Then you could either make a PR with your change, or I can do it (up to you).

from ansible-docker.

If you decide you want to do the PR, you can do that before then if you want and I'll merge it after I test it here.

from ansible-docker.

I had to act fast because Docker 18.09 shipped and they changed something big when it comes to their systemd unit file. Long story short, this role was broken by default with 18.09.

v1.5.0 is now pushed which fixes the issue in the following way:

1. This role now apt installs python-pip by default.
2. This role now pip installs docker always.

The reason I went for "always" instead of just when at least 1 Docker login was set was because you might want to use docker_service without logging in, in which case, the pip docker package needs to be installed.

In the future, perhaps the pip install step could be turned into a variable which is also tied into installing python-pip in the case where you don't use Docker login or the service module and want to avoid installing both of those packages.

from ansible-docker.

Fantastic! Really impressed by your reactivity!

from ansible-docker.