thomvaill / tads-boilerplate Goto Github PK

Describe the solution you'd like
Persistent storage is a hassle with Docker Swarm, and is not covered at all in this project for now.
I propose to integrate REX-Ray installation and configuration to the playbooks.
It supports all famous cloud providers and Vagrant!
So we would be able to test it into the vagrant environment.
On localhost we would use classic volumes.

Describe alternatives you've considered

• cloudstor for docker-for-aws
• cloudstor docker-for-azure

... but those plugins are not open sourced and are cloud provider specific.

Additional context
Developing this feature would enable us to add more examples / pre-setupped services:

• portainer
• swarmprom
• ELK

Possible caveats:

• REX-Ray EBS for AWS is mono-AZ (contrary to cloudstor)

Describe the bug
Im up to this command from your example
> ./tads ansible-playbook localhost provision

Tasks are executing fine up to this one:

PLAY [docker:&dockerswarm_manager] *************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************ok: [127.0.0.1]

TASK [load docker info as facts] ***************************************************************************************************************************************An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: Value of unknown type: <class 'urllib3.exceptions.ProtocolError'>, ('Connection aborted.', error(2, 'No such file or directory'))
fatal: [127.0.0.1]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1575456373.86-208554238811029/AnsiballZ_docker_info_facts\", line 102, in <module>\n
 _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1575456373.86-208554238811029/AnsiballZ_docker_info_facts\", line 94, in _ansiballz_main\n
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n
File \"/root/.ansible/tmp/ansible-tmp-1575456373.86-208554238811029/AnsiballZ_docker_info_facts\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.docker_info_facts', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python2.7/runpy.py\", line 188, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n
File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/modules/docker_info_facts.py\", line 78, in <module>\n
File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/modules/docker_info_facts.py\", line 69, in main\n  File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/module_utils/basic.py\", line 2093, in fail_json\n  File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/module_utils/basic.py\", line 2065, in _return_formatted\n  File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/module_utils/basic.py\", line 418, in remove_values\n
File \"/tmp/ansible_docker_info_facts_payload_5ZDtsE/ansible_docker_info_facts_payload.zip/ansible/module_utils/basic.py\", line 401, in _remove_values_conditions\nTypeError: Value of unknown type: <class 'urllib3.exceptions.ProtocolError'>, ('Connection aborted.', error(2, 'No such file or directory'))\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Any Idea how where to look at?

To Reproduce
Steps to reproduce the behavior:
> ./tads ansible-playbook localhost provision

Expected behavior
A clear and concise description of what you expected to happen.

Environment (please complete the following information):

• OS/distro: Latest Ubuntu docker image, on windows Docker Desktop

Additional context
Add any other context about the problem here.

Describe the bug
I cant get letsencrypt certificates for subdomains.

To Reproduce
I have the A records of the subdomains pointing to the Server.
In the stack yaml file i added the known subdomains:

traefik.frontend.rule=Host:{{ domains.main }},{{ domains.main_www }},{{ domains.eco_ivobathke }},{{ domains.oxid_ivobathke }}

The letsencrypt block in traefik.yml.j2 is:

{% if letsencrypt %} # you can also use conditional statements
      - --acme
      - --acme.acmeLogging
      - --acme.storage=traefik/acme/account
      - --acme.entryPoint=https
      - --acme.email=admin@{{ domains.main }}
      - --acme.httpchallenge.entrypoint=http
      - --acme.domains={{ domains.main }},{{ domains.main_www }}
{% endif %}

I am reading about defining main & sans domain in the traefik.toml file.
https://doc.traefik.io/traefik/v1.7/configuration/acme/#domains
But i dont see how to use this here, maybe to switch to toml config instead of commands with the traefik image?

Checking the logs i see:

Datastore sync error: object lock value: expected ...

Seems related to: traefik/traefik#3487

Expected behavior
Traefik should automatically request letsencrypt certificates for known subdomains.

Describe the bug
Running the command ./tads install-dependencies generates the following error:

TASK [Install Terraform (linux)] ***************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (ansible.legacy.command) module: warn. Supported parameters include: _raw_params, _uses_shell, argv, chdir, creates, executable, removes, stdin, stdin_add_newline, strip_empty_ends."}

To Reproduce
Steps to reproduce the behavior:

1. Clone the project from master
2. Install dependencies

Expected behavior
Dependencies are installed without error.

Environment (please complete the following information):

• OS/distro: Ubuntu 22.04.2 LTS
• Ansible version: 2.14.4
• Terraform version: N/A
• Cloud provider: N/A
• Vagrant version: N/A

Additional context
Appears to be related to this StackOverflow question.

Following changes fixes the issue:

diff --git a/ansible/install-dependencies.yml b/ansible/install-dependencies.yml
index 619743d..6b828a5 100644
--- a/ansible/install-dependencies.yml
+++ b/ansible/install-dependencies.yml
@@ -47,7 +47,7 @@
         TERRAFORM_VERSION: 0.12.12
       args:
         creates: ~/.local/bin/terraform
-        warn: False
+          # warn: False
       when: ansible_facts['os_family'] != "Darwin"

     - name: Install Terraform Homebrew Tap (Mac)

As mentioned in the README, I did not have time to develop this feature properly yet.

I used to do CD with a T.A.D.S. similar repository by (roughly):

• Always referer to Docker images with tags
• Manage those tags into a separate tags.yml file
• Write a script that can update a tag, perform a ./tads ansible-playbook production deploy, and in case of success commit and push the file
• Make the CI/CD tool use this script in the deployment pipeline

We may also consider the option of using Portainer's API to achieve CD.
But I'm not really in favor of this because the main advantage of commiting the newly deployed tags is that all developers can stay in sync with the prod.

Don't know what ADR is? -> https://adr.github.io
Since it's an IaC repository, this is the ideal place to store ADRs!

For this issue, we should test and choose the best tool to integrate.

Describe the bug
setup fails on ./tads ansible-playbook localhost provision

reports that pip is not found - apperas to be an error with the gerrlingguy.pip role. Solved this issue by updating the
requirements.yaml to the following:

###
# ansible-galaxy requirements
##
# Automatically installed before each T.A.D.S. Ansible command
##

- src: geerlingguy.pip
  version: 2.1.0

- src: geerlingguy.docker
  version: 3.1.2

To Reproduce
Follow the exact steps in the README.

Expected behavior
Expected deployment to work as shown in demo video.

Environment (please complete the following information):

• OS/distro: Ubuntu 20.04 desktop
• Ansible version: 2.9.6
• Terraform version: v0.15.5
• Cloud provider: n/a
• Vagrant version: 2.2.6

Additional context
note the default version of terraform installed by the script is now outdated.

Because having to type the whole commands is a pain!

Those services should be pre-installed in the example so it's quicker to bootstrap a project:

• portainer for Swarm management, RBAC, API...
• swarmprom for the monitoring
• ELK for logs management

However, those services require persistent storage. So we are blocked by #3.

Make the project work on OS X.

Since I don't have a Mac, help is welcome :-)

For this issue, we have to consider to potentially dockerize the CLI + Ansible + Terraform.
However I don't think dockerizing Vagrant is possible.
We have to think about the best option to do this.

Since DO is really simple to use, we should add a Terraform example.