Enhancement: Test with TestInfra and Molecule about ansible-collection-hardening HOT 7 CLOSED

Building on @samrocketman's point, a nice alternative testing framework (similar to testinfra or serverspec) is GOSS. It's super lightweight, tests are easy to develop (just a YAML file), and it's REALLY fast (much faster in my experience than both serverspec and testinfra).

from ansible-collection-hardening.

If the dev-sec team is open to this idea I'm willing to create a PR with suggested changes. This assumes #127 is an accepted idea because all of the technologies discussed in this issue are installed via Python pip.

from ansible-collection-hardening.

We're using inspec for all our tests (see https://github.com/dev-sec/linux-baseline/) and since we use these for the chef, puppet and ansible roles, its unlikely that we will replace them with TestInfra.

However some (long) time ago I took a look at molecule and found it quite good. However there's no support for inspec so it's out of the question right now. However @chris-rock thought about writing a driver for molecule to support inspec.

So if that happens, we'll see gladly take a PR that implements molecule testing.

from ansible-collection-hardening.

@samrocketman Thank you very much for your open feedback. I'd like to understand your concerns a little bit more about. Could you elaborate on:

• Why is Ruby as a dependency for testing an issue (it is not a runtime dependency)?
• How is the workflow of Molecule different from test-kitchen?
• Why do you think testinfra is better then InSpec?

from ansible-collection-hardening.

Why is Ruby as a dependency for testing an issue (it is not a runtime dependency)?

It's not really a concern. It just simplifies setting up a development and test environment with fewer dependencies. For example, really the only dependencies you need installed is Python, pip, and virtualenv. The rest of the dependencies would get installed via requirements.txt (akin to Ruby Gemfile.lock).

How is the workflow of Molecule different from test-kitchen?

I have only a light familiarity with test kitchen. However, conceptually they're similar. The workflow should be somewhat the same.

1. Develop ansible task and the test.
2. Run molecule test which in one command: provisions, installs ansible if missing, runs the playbook, evaluates the result with testinfra tests, reports the results. I'm not sure if deprovisioning happens before or after reporting.

Why do you think testinfra is better then InSpec?

Because Ansible, molecule, and TestInfra are all tracked by python, it's easy to track them all for repeatability within requirements.txt.

I would say that's the primary advantage but it's not a big one. You can achieve the same level of repeatability with Gemfile.lock. It's just a fewer set of dependencies.

Keep in mind I'm not suggesting test kitchen and inspec are inferior. They both can achieve the same thing with success. I'm only suggesting simplifying your workflow with fewer dependencie and a tool specifically designed with ansible in mind.

from ansible-collection-hardening.

Here's an example with the dependency chain I'm recommending. https://github.com/Comcast/ansible-sdkman/blob/master/.travis.yml

from ansible-collection-hardening.

Closing this for now. If anyone wants to support other testing-methods, feel free to reopen and provide a PR.

from ansible-collection-hardening.