Correctly handle ANSIBLE_FILTER_PLUGINS for plugin vendored filter functions about molecule-hetznercloud HOT 7 CLOSED

Hello, coworker of @ggggut here. I can reproduce this in a virtualenv.
I don't 100% know how molecule drivers are supposed to work, mostly it is just guessing and looking at existing drivers. But after looking more at other drivers, especially https://github.com/ansible-community/molecule-docker (which also uses a filter plugin), I think the create.yml and destroy.yml playbooks are not supposed to be in the cookiecutter template. This way they are not templated during the role creation and instead the original playbooks of the driver are used, which are able to find the filter plugin at the correct location.
I tested this solutions locally and everything seems to work just fine. @decentral1se do you think this is the proper solution or am I missing something that requires the files to be templated as part of the role creation? If you give your okay I can provide a pull request.

from molecule-hetznercloud.

Hi @tumbl3w33d! Thanks for taking some time on this one.

I'd suggest to add a failed_when that catches this expected condition.

💯 will happily merge a PR for this 💯

A second thing that caused problems was the generated create.yml as it uses the filter molecule_get_hetznercloud_networks that are not in the filter_path and there's no explanation in the documentation how to fix that.

This sounds like a bug. @ggggut do you have any idea why that might be happening (from #29). I'll note that this doesn't happen in the test suites so I am wondering if this is something in your local setup @tumbl3w33d but I can't be quite sure what. I don't have much time this week but I'll try to look into this shortly. If you have any fix suggestions, let me know!

from molecule-hetznercloud.

Thank you for your fast reply. I'll try to gather some more in-depth information about what causes the filter path issue.

from molecule-hetznercloud.

I reproduced it in a fresh folder with a fresh role:

# created a new directory 'foo' and stepped into it
# then we create a virtualenv
[myuser@myworkstation foo]$ python3 -m venv venv
# and activate it
[myuser@myworkstation foo]$ . venv/bin/activate
# we install the modules ansible, molecule[hetznercloud] and molecule-hetznercloud
(venv) [myuser@myworkstation foo]$ pip install -r requirements.txt
[…]
# we create a fresh role
(venv) [myuser@myworkstation foo]$ molecule init role foo -d hetznercloud
INFO     Initializing new role foo...
No config file found; using defaults
- Role foo was created successfully
INFO     Initialized role in /home/myuser/projects/foo/foo successfully.

Then I prepared the run:

• set HCLOUD_TOKEN via direnv file
• disable linting in molecule.yml because it's irrelevant here and keeps from reaching the right playbook
• step into the role folder with cd foo
• molecule --debug test

  TASK [Create private network(s)] ***********************************************
fatal: [localhost]: FAILED! => {"msg": "template error while templating string: No filter named 'molecule_get_hetznercloud_networks'.. String: {{ molecule_yml.platforms|molecule_get_hetznercloud_networks('networks') }}"}

• debug output: ANSIBLE_FILTER_PLUGINS=/home/myuser/projects/foo/venv/lib/python3.9/site-packages/molecule/provisioner/ansible/plugins/filter:/home/myuser/.cache/molecule/foo/default/plugins/filter:/home/myuser/projects/foo/foo/plugins/filter:/home/myuser/.ansible/plugins/filter:/usr/share/ansible/plugins/filter
• actual path: /home/myuser/projects/foo/venv/lib/python3.9/site-packages/molecule_hetznercloud/playbooks/filter_plugins/get_hetznercloud_networks.py

Not sure yet if I did something wrong at some point, but the path of the virtualenv is different from where molecule is searching.

from molecule-hetznercloud.

Hey @ekeih, thanks for looking into this! So, we need to pass our own create.yml because we have specific Hetzner Cloud logic we need to take care of (like creating networks). You can see an example of another drive which is doing this in https://github.com/ansible-community/molecule-vmware/tree/master/molecule_vmware/cookiecutter/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D.

From the good investigation of @tumbl3w33d it looks like we instead need to hook into the ANSIBLE_FILTER_PLUGINS logic to make sure the correct path is available. Back in upstream Molecule, that is defined like so:

https://github.com/ansible-community/molecule/blob/3d3ce253f396ac0117659bf9f2cbc06c7377b508/src/molecule/provisioner/ansible.py#L461

We can hook into that in our own subclass here in this plugin over at:

We need to get something like /home/myuser/projects/foo/venv/lib/python3.9/site-packages/molecule_hetznercloud/playbooks/filter_plugins/ onto this path while preserving the existing paths?

Does anyone have energy/time for a PR for this?

from molecule-hetznercloud.

I saw that an active virtualenv comes with an env var that we can rely on to build the path:

VIRTUAL_ENV=/home/myuser/projects/somepath/myproject/venv

Not sure how to handle the moving python version, though.

Edit: $VIRTUAL_ENV/lib/*/site-packages/molecule_hetznercloud/playbooks/filter_plugins using glob magic would do the trick. Might need to handle multiple python folders appearing there and add all of them

from molecule-hetznercloud.

Thanks for the additional insights @decentral1se :)

I understand that we need our own create.yml, but I think we don't need it in the driver in .../site-packages/molecule_hetznercloud/... and in the templated role we get by running molecule init role foo -d hetznercloud. If I understand the code correctly there is nothing special templated by cookiecutter into the file, it is just a raw copy of the same code that is also installed with the driver.

I created #33 to better show what I mean. If I use the code from this branch everything works as expected. Maybe you can try the branch and check if it also works for you or if I am missing a scenario where the templated copy of the playbooks is required.

from molecule-hetznercloud.