renskiy / fabricio Goto Github PK

The idea is to forward outgoing registry requests (80 and 443 ports) to SSH tunnel with proxy in transparent mode (e.g. squid) on the other side of tunnel.

Forwarding can be set up with iptables:

iptables --table nat --append OUTPUT --protocol tcp --dport 80 --dport 443 --jump DNAT --to-destination 127.0.0.1:3128

Transparent proxy may be started on the admin side as a Docker container.

PS: don't forget to reset iptables rules to original state after all. This can be reached by following steps: reset => set => reset, where first step need to be sure there is no any rules from previous (possibly failed) deploy.

There is at least one place where used bash-specific command which can not be executed by default on Windows: DockerTasks.delete_dangling_images().

There is native rollback implementation available for Docker services which can be used by default.

During installation encountered the following error:

fabricio.operations.Error: local() encountered an error (return code 1) while executing 'docker push...

Environment:
Mac OS 10.x
Fabric==1.14.0
fabricio==0.5.6

denied: requested access to the resource is denied

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/fabric/main.py", line 757, in main
*args, **kwargs
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 426, in execute
results[''] = task.run(*args, **new_kwargs)
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 173, in run
return self.wrapped(*args, **kwargs)

...

and get DockerTasks.pull() print pulling statistics

maybe it should just execute this local command: docker images | sed 1d | grep '<none>' | awk '{print($3)}' | xargs docker rmi

Thank you for the tool, Rinat.

Why doesn't it support a case of using docker-compose file on a server, but without swarm?

It might be useful with https://github.com/wemake-services/wemake-django-template for a single node scenario.

If the case is possible, please, describe how it could be configured.

Hi.
I tried to use fabricio to deploy my django project to prod.
I used docker and docker-compose before on local so I created docker.Stack service with link to my docker-compose file.
But the problem is that fabricio copies docker-compose.yml to /tmp and looks for all files, linked in docker-compose.yml, in this directory. For example, my docker-compose.yml has line:

env_file:
          - docker/dev.env

And at building container, docker looks for env file in /tmp/docker/dev.env and I get error.
What is worse, dockerfiles are not found too.
I tried:

1. Create docker.Stack with temp_dir parameter
2. Subclass Stack and point temp_dir to my project directory (and re-define upload_configuration_file)
3. Create symlink in tmp to my project directory.
   Without any success.
   Did I miss something important?

why is vagrant needed? can I deploy it without it i.e. to digital ocean?

can you please add documentation or example on how to deploy docker image from docker-hub to digital ocean?

Or add such option to DockerTasks

need support for python3

For now fabric.api.run and fabric.api.local are used to run Docker commands. It is better to use docker-py's Docker client instead.

There is a bunch of new removable options added to the Docker services recently:

• config
• dns
• dns-option
• placement-pref
• group
• host
• secret

Hi
How can I remove containers and services related to a task.

In situations when using custom registry with PullDockerTasks, remote_tunnel can take port which already used by some of containers. Therefore, PullDockerTasks should detect if remote_tunnel is really necessary.

In most cases remote_tunnel is necessary only for registries which point to the local host.

This behaviour should fix fab --display <task>.

Same way as #127 (Docker stacks / Docker Compose) was resolved there is opportunity to add Kubernetes support.

Hi, what could the source of the problem?
Beating the head against a wall

Fabric==1.14.0
fabricio==0.3.1

local: docker build --tag front:latest --pull --build-arg nginx_config_filename=local.conf .
Sending build context to Docker daemon 436.6MB
Step 1/15 : FROM centos:centos7
Get https://registry-1.docker.io/v2/: EOF

Warning: local() encountered an error (return code 1) while executing 'docker build --tag front:latest --pull --build-arg nginx_config_filename=local.conf .'

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/fabric/main.py", line 757, in main
*args, **kwargs
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 426, in execute
results[''] = task.run(*args, **new_kwargs)
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 173, in run
return self.wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/fabricio/tasks.py", line 368, in deploy
fab.execute(self.prepare, tag=tag, no_cache=no_cache)
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 426, in execute
results[''] = task.run(*args, **new_kwargs)
File "/usr/local/lib/python2.7/site-packages/fabric/tasks.py", line 173, in run
return self.wrapped(*args, **kwargs)
File "/Users/myusername/Develop/chat-bot/frontend/fabfile.py", line 206, in prepare
use_cache=True,
File "/usr/local/lib/python2.7/site-packages/fabricio/init.py", line 74, in local
**kwargs
File "/usr/local/lib/python2.7/site-packages/fabricio/init.py", line 27, in _command
raise RuntimeError(result)
RuntimeError

It would be very useful to add support of Docker stacks. This also will add support of Docker Compose to Fabricio.

I think it would be nice to have ProvisionTasks inheriting base Tasks and providing some basic provisioning commands. Such as:

• init (default)
• remove
• update

Such class would be useful for management of Docker networks, volumes and other entities.

can I start 4 running versions of same docker image?

I see a replicas option, can It be used to spin 3 dockers

It is good idea to check if local registry is alive before doing large-weight operations (like pull from original registry or build)

Need command that removes service.

Any attempt to execute deploy configuration fails with following message:

RuntimeError: local() encountered an error (return code 1) while executing 'for img in $(docker images --filter "dangling=true" --quiet); do docker rmi "$img"; done'

This happens when Docker can't delete some images (due to existing containers from such images, for example).

There are two alternatives if Fabric can't determine host for task:

1. Fabric can ask host using prompt
2. use --abort-on-prompts option

Both cases lead to inability to complete automatic scripts based on Fabricio. It's necessary to have graceful skipping of tasks which's host can't be determined at runtime.

Ubuntu 17.10
python2, python3
Fabric 1.13.1
Paramiko 2.0.0
fabricio Version: 0.5.6

Running fab -l causes

AttributeError: 'module' object has no attribute 'BuildDockerTasks'

Is it any way how launch containers locally?

I have tried to create

# Infrastructure
@fabricio.tasks.infrastructure
def localhost():
    fab.env.roledefs.update(
        amqp=['anton@localhost'],
        reverse_proxy=['anton@localhost'],
        postgres=['anton@localhost'],
        tarantool=['anton@localhost'],
        backend=['anton@localhost'],
    )

but got error

Fatal error: Low level socket error connecting to host localhost on port 22: Unable to connect to port 22 on 127.0.0.1 (tried 1 time)

Underlying exception:
    Unable to connect to port 22 on 127.0.0.1

Собираю образ, пытаюсь его запулить в локал регистри (у меня потом на базе этого образа собирается основной)
.pull не срабатывает, так как не указан хост в таске, но мне же хост и не нужен, registry то указан, я хочу просто сохранить образ, а следующий таск уже соберет основной на его базе и запулит его на бой.

'pull' execution was skipped due to no host provided (command: edcontainer.pull)

Можно ли подумать над изменением этой логики?

container.py

_exec_command = 'docker exec --tty --interactive {container} {command}'_

I have a need to run commands with the --detach option

There is 'ports' property in the fabricio.docker.Container but lack of 'expose'.

Options, commands, arguments, etc.

Values should be wrapped by quotes. All special symbols should be escaped.

RuntimeError: Conflicting options: --restart and --rm

I cannot upload a compose-file from a folder other than the fabfile.py folder since the "upload_configuration_file" script will try to upload it to the path of the mentioned local folder.
Example:
stack = tasks.DockerTasks(
service=docker.Stack(
name='stack',
options={
'compose-file':'/folder0/../foldern/docker-compose.yml',
},
),
hosts=fab.env.hosts,
destroy_command=True,
)

fab will try to upload the docker-compose file to "/folder0/../foldern/docker-compose.yml" and not to temp_dir folder which will cause error if the path does not exist in the distant machine

Version of fabricio==0.3.1

To force deploy locally I started describing the task as following

@tasks.infrastructure
def localhost():
    print("Trying to run fab locally")
    fabricio.run = functools.partial(fabricio.local, capture=True)
    fab.env.update(
        roledefs={
            'queue': ['localhost'],

...

Still it had no effect, it tries to deploy locally via ssh

I need to use latest fabric version features i.e. user and key_file in Task. is it possible to support latest fabric?

This property can be used to assign custom commands name for groups of tasks, including infrastructures. For example:

name_with_underscores = Tasks(name='name-with-hyphens')

or

@infrastructure(name='UPPER')
def lower()
    pass