docker / machine Goto Github PK

Machine management for a container-centric world

Home Page: https://docs.docker.com/machine/

License: Apache License 2.0

Makefile 0.69% Go 93.77% Shell 5.50% Dockerfile 0.04%

machine's Introduction

Docker Machine

Machine lets you create Docker hosts on your computer, on cloud providers, and inside your own data center. It creates servers, installs Docker on them, then configures the Docker client to talk to them.

It works a bit like this:

$ docker-machine create -d virtualbox default
Running pre-create checks...
Creating machine...
(default) Creating VirtualBox VM...
(default) Creating SSH key...
(default) Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect Docker to this machine, run: docker-machine env default

$ docker-machine ls
NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER   ERRORS
default   -        virtualbox   Running   tcp://192.168.99.188:2376           v1.9.1

$ eval "$(docker-machine env default)"

$ docker run busybox echo hello world
Unable to find image 'busybox:latest' locally
511136ea3c5a: Pull complete
df7546f9f060: Pull complete
ea13149945cb: Pull complete
4986bf8c1536: Pull complete
hello world

In addition to local VMs, you can create and manage cloud servers:

$ docker-machine create -d digitalocean --digitalocean-access-token=secret staging
Creating SSH key...
Creating Digital Ocean droplet...
To see how to connect Docker to this machine, run: docker-machine env staging

$ docker-machine ls
NAME      ACTIVE   DRIVER         STATE     URL                         SWARM   DOCKER   ERRORS
default   -        virtualbox     Running   tcp://192.168.99.188:2376           v1.9.1
staging   -        digitalocean   Running   tcp://203.0.113.81:2376             v1.9.1

Installation and documentation

Full documentation is available here.

Contributing

Want to hack on Machine? Please start with the Contributing Guide.

Driver Plugins

In addition to the core driver plugins bundled alongside Docker Machine, users can make and distribute their own plugin for any virtualization technology or cloud provider. To browse the list of known Docker Machine plugins, please see this document in our docs repo.

Troubleshooting

Docker Machine tries to do the right thing in a variety of scenarios but sometimes things do not go according to plan. Here is a quick troubleshooting guide which may help you to resolve of the issues you may be seeing.

Note that some of the suggested solutions are only available on the Docker Machine master branch. If you need them, consider compiling Docker Machine from source.

`docker-machine` hangs

A common issue with Docker Machine is that it will hang when attempting to start up the virtual machine. Since starting the machine is part of the create process, create is often where these types of errors show up.

A hang could be due to a variety of factors, but the most common suspect is networking. Consider the following:

Are you using a VPN? If so, try disconnecting and see if creation will succeed without the VPN. Some VPN software aggressively controls routes and you may need to manually add the route.
Are you connected to a proxy server, corporate or otherwise? If so, take a look at the --no-proxy flag for env and at setting environment variables for the created Docker Engine.
Are there a lot of host-only interfaces listed by the command VBoxManage list hostonlyifs? If so, this has sometimes been known to cause bugs. Consider removing the ones you are not using (VBoxManage hostonlyif remove name) and trying machine creation again.

We are keenly aware of this as an issue and working towards a set of solutions which is robust for all users, so please give us feedback and/or report issues, workarounds, and desired workflows as you discover them.

Machine creation errors out before finishing

If you see messages such as "exit status 1" creating machines with VirtualBox, this frequently indicates that there is an issue with VirtualBox itself. Please file an issue and include a link to a Github Gist with the output of the VirtualBox log (usually located at $HOME/.docker/machine/machines/machinename/machinename/Logs/VBox.log), as well as the output of running the Docker Machine command which is failing with the global --debug flag enabled. This will help us to track down which versions of VirtualBox are failing where, and under which conditions.

If you see messages such as "exit status 255", this frequently indicates there has been an issue with SSH. Please investigate your SSH configuration if you have one, and/or file an issue.

"You may be getting rate limited by Github" error message

In order to create or upgrade virtual machines running Docker, Docker Machine will check the Github API for the latest release of the boot2docker operating system. The Github API allows for a small number of unauthenticated requests from a given client, but if you share an IP address with many other users (e.g. in an office), you may get rate limited by their API, and Docker Machine will error out with messages indicating this.

In order to work around this issue, you can generate a token and pass it to Docker Machine using the global --github-api-token flag like so:

$ docker-machine --github-api-token=token create -d virtualbox newbox

This should eliminate any issues you've been experiencing with rate limiting.

machine's People

Contributors

Stargazers

Watchers

Forkers

jpgriffo psftw rtvt123 cedarlogic legal90 lalyos wallnerryan ehazlett dlorenc andrewwatson alexislitool svendowideit wjin bfirsh nishanthvasudevan josephwinston smashwilson exoscale rayleyva dublado ertemplin nathanleclaire exadeci arminc foliea waitingkuo misteritguru worldline baseboxorg atsaki liudch pmoust cpuguy83 scalp42 ggiamarchi ifa6 hugoduncan unicell afey cjellick bastichelaar mtsuk tlambe lemonhall david-richer-adsk guilhermebr louisdex jbayer sujithvellat wangnan8791 saturnism huskyeder milliant mattxia rgbkrk mbrukman direktspeed tebro taoh emilevauge titanlien punchmk jaharmi itraor bettisonr robertbreker akirill0v kkboy123 stefanfoulis michje r4mi5 riddopic ianblenke dougborg tonyhb wwwjfy rickard-von-essen centurylinklabs freiit cortezone chanezon stackengine iij emanuelemazza roldancer hspencer77 s7v7nislands aykit maximedevalland mohitsoni jmcarbo pcittadini ameerrafiq adam-zacharski wescravens jamesclonk maounis varmamahesh kgrvamsi stevesmename

machine's Issues

Machine storage directory should be configurable

You might not want to put your machines in .docker/hosts.

This should probably also be .docker/machines or .machine/machines by default.

Error on first run "public-key.json: no such file or directory"

I just downloaded machine binary to /usr/local/bin on my Mac (OS.X 10.10 Yosemite) and running machine create -d virtualbox test, with VirtualBox already installed, leads to error FATA[0057] open /Users/paris/.docker/public-key.json: no such file or directory.

Could you please help me out with that?

Logs

INFO[0000] Downloading boot2docker...                   
INFO[0014] Creating SSH key...                          
INFO[0015] Creating VirtualBox VM...                    
INFO[0024] Starting VirtualBox VM...                    
INFO[0024] Waiting for VM to start...                   
FATA[0057] open /Users/paris/.docker/public-key.json: no such file or directory

Warn about Virtualbox not being available sooner

$ machine create -d virtualbox dev
INFO[0000] Downloading boot2docker...
INFO[0083] Creating SSH key...
INFO[0083] Creating VirtualBox VM...
FATA[0083] exec: "VBoxManage": executable file not found in $PATH

In general (I think) if something a driver needs/depends-on isn't available it would be great if the user was told about it sooner.

Proposal: Add Support Driver for GCE instances

I'd like to investigate the feasibility of adding a driver for creating docker machines in Google Compute Engine through the Google Cloud APIs. I have some experience with these APIs and with "GKE" and kubernetes so I'd be glad to contribute to / lead this effort.

Set up machines using cloudinit

This will be far more reliable than logging into machines with SSH to set them up.

This doesn't need to be in the initial iteration of this, but it should also be customisable. This will make it really easy for users to customise what machine does to provision a machine. Also means we can support things which aren't Ubuntu because you can provide a cloudinit config for whatever image you would rather use.

"machine" command name conflict

OS X (and I assume other BSD variants) already has a "machine" command line tool: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/machine.1.html

Probably best not to conflict with it.

Documentation for 1.0

Upon release of a stable version of Machine, we will add some docs to https://docs.docker.com/.

This should be things like:

Installation
Getting started
Documentation for specific drivers
CLI documentation

DigitalOcean bugs

(machine ls) goes out of sync with the machines listed at DO's web GUI. Specifically, (machine) assumes droplets are dead when they are still alive. How hard is it to hit the API properly?
No obvious way to affect DO host's configuration - hosts are set up without SSH access (though you can reboot from web GUI to set a root password). First thing that became an issue was that DO's default "no swap" configuration on the host seemed to cause MySQLd to fail upon startup inside a container when it tried to mmap 128MB of InnoDB buffer, on a 512MB host.
Random failures of communication between (machine) and DO, but this happens on Tugboat too. No surprises there.

This was all done on the SG region. I'm in MY.

Amazon driver should support availability zone

Right now we support --amazonec2-region but there's no --amazonec2-availability-zone.

[Feature Request] struct tag for secrets

Currently the drivers define a Go struct which maps to what gets stored on disk for that host. Since this is tied directly to the information demarshalled from command line flags used to launch a host, it includes information such as API access tokens which is not needed to be stored on disk and increases the risk profile for users.

I propose a struct tag to solve this problem. It could be called "secret" or something similar, and would indicate to machine that the field should not be copied to disk when the configuration information is stored.

"url" command does not return host url

When running without args, the command does not return the url:

ehazlett@ejh-mbp ~/Sync> ./machine create -d virtualbox machine-test
INFO[0000] Downloading boot2docker...
INFO[0005] Creating SSH key...
INFO[0005] Creating VirtualBox VM...
INFO[0010] Starting VirtualBox VM...
INFO[0011] Waiting for VM to start...
INFO[0046] "machine-test" has been created and is now the active machine. To point Docker at this machine, run: export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity
ehazlett@ejh-mbp ~/Sync> ./machine ls
NAME           ACTIVE   DRIVER       STATE     URL
machine-test   *        virtualbox   Running   tcp://192.168.99.100:2376
ehazlett@ejh-mbp ~/Sync> ./machine url
NAME:
   url - Get the URL of a machine

USAGE:
   command url [arguments...]

active machine

I seem to have to update my DOCKER_HOST entry manually every time I change active machine.

I got the impression that machine would take care of this for me. I know it can't change my environment, but was expecting it to make docker find the host address in some other way.

The last part of the last line when creating a new machine is particularly misleading now:

$ machine create -d virtualbox dev
INFO[0000] Downloading boot2docker...                   
INFO[0042] Creating SSH key...                          
INFO[0043] Creating VirtualBox VM...                    
INFO[0058] Starting VirtualBox VM...                    
INFO[0059] Waiting for VM to start...                   
INFO[0090] "dev" has been created and is now the active machine. Docker commands will now run against that machine.

As it claims that my docker commands is now going to the newly created machine, when in fact it does not, until I run export DOCKER_HOST=$(machine url).

What am I missing, or is it just a missing feature that's not yet implemented.. ?

boot2docker on Digital Ocean

I'd like to see boot2docker on Digital Ocean :)

I know DO themselves don't support custom images; but they do obviously accept requests to add additional "supported" images. Since CoreOS (IHMO) is heading down a different path it would be good if the "docker" image on DO was actually bootdocker (again IHMO).

--n-instances flag for machine create

Suppose I want to use docker-machine to spin up a simple architecture of two instances running the same application behind another instance which serves as a proxy / load balancer to the two. It'd be nice to spin up the application instances using something like:

$ machine create --driver ec2 --n-instances 2 load_balanced_app
...
$ machine ls
NAME                  ACTIVE   DRIVER   STATE     URL
default                                           unix:///var/run/docker.sock
load_balanced_app_1   *        ec2      Running   tcp://54.183.69.81:2375
load_balanced_app_0            ec2      Running   tcp://54.183.67.108:2375

I know that the Amazon API allows you to specify the number of instances desired to the RunInstances call, I'd imagine that similar args exist for other providers. If not, they can probably be worked around by making multiple calls.

[DigitalOcean driver] Don't run as root

The DigitalOcean driver user is root by default. This should be a lower-privileged user instead.

[VBox] Everything works great until VM reboot, then I get TLS Handshake error: remote error: bad certificate

$ machine create -d virtualbox dev
INFO[0000] Downloading boot2docker...
INFO[0056] Creating SSH key...
INFO[0056] Creating VirtualBox VM...
INFO[0063] Starting VirtualBox VM...
INFO[0063] Waiting for VM to start...
INFO[0094] "dev" has been created and is now the active machine. To point Docker at this machine, run: export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity

$ machine ls
NAME   ACTIVE   DRIVER       STATE     URL
dev    *        virtualbox   Running   tcp://192.168.99.116:2376

$ export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity

$ docker-auth ps
The authenticity of host "192.168.99.116:2376" can't be established.
Remote key ID ACQA:AGRZ:5BLY:YDGS:NKPQ:ITBH:5JKA:QQXF:B5I4:KCXE:GCGJ:TRS7
Are you sure you want to continue connecting (yes/no)? yes
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

$ docker-auth version
Client version: 1.3.1-dev
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 831d09f
OS/Arch (client): darwin/amd64
Server version: 1.3.1-dev
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 831d09f

$ docker-auth run busybox echo hello world
Unable to find image 'busybox:latest' locally
busybox:latest: The image you are pulling has been verified

511136ea3c5a: Pull complete
df7546f9f060: Pull complete
e433a6c5b276: Pull complete
e72ac664f4f0: Pull complete
Status: Downloaded newer image for busybox:latest
hello world

Ok, everything works great. Great! Until....

$ machine stop
$ machine start
INFO[0000] Waiting for VM to start...
$ docker-auth ps
FATA[0000] TLS Handshake error: remote error: bad certificate
$ docker-auth version
FATA[0000] TLS Handshake error: remote error: bad certificate
$ docker-auth run busybox echo hello world
FATA[0000] TLS Handshake error: remote error: bad certificate

I am using the auth-enabled binary (referred to as docker-auth above) provided in the README from OSX.

Make VirtualBox network configurable

At the moment the subnet used by VirtualBox is hard coded. It would be neat if this was configurable, like in boot2docker-cli.

FATA: /v1.16/containers/create: x509: certificate is valid for 0.0.0.0, not 192.x.x.x

Hi there,

I'm following the tutorial and I got everything setup...

Setup

$ boot2docker version
Boot2Docker-cli version: v1.4.1
Git commit: 43241cb

$ boot2docker ssh
                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.4.1, build master : 86f7ec8 - Tue Dec 16 23:11:29 UTC 2014
Docker version 1.4.1, build 5bc2ff8

I created the credentials using Docker 1.3.1, as described:

mdesales@SDGl121456EB ~/Downloads (mac) $ ls -la ~/.docker/
hosts/           key.json         public-key.json

Also, I have installed machine, and created the "dev" machine

$ machine ls
NAME   ACTIVE   DRIVER       STATE     URL
dev    *        virtualbox   Running   tcp://192.168.99.102:2376
mdesales@SDGl121456EB ~/Downloads (mac) $ machine inspect dev
{
    "DriverName": "virtualbox",
    "Driver": {
        "MachineName": "docker-host-8b23bfcad60cc1ca0db5e80df0525c079e6b58ab8c4ba27130019af9bef11efb",
        "SSHPort": 54733,
        "Memory": 1024,
        "DiskSize": 20000,
        "Boot2DockerURL": ""
    }
}

Problem

I created the machine as directed, and it mentions about the creation of the SSH keys...

$ machine create -d virtualbox dev
INFO[0000] Downloading boot2docker...                   
INFO[0010] Creating SSH key...                          
INFO[0010] Creating VirtualBox VM...                    
INFO[0015] Starting VirtualBox VM...                    
INFO[0015] Waiting for VM to start...                   
INFO[0046] "dev" has been created and is now the active machine. To point Docker at this machine, run: export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity

I did run the command above and I can't get int...

 $ machine ls
NAME   ACTIVE   DRIVER       STATE     URL
dev    *        virtualbox   Running   tcp://192.168.99.102:2376

$ docker run busybox echo hello world
FATA[0000] An error occurred trying to connect: Post https://192.168.99.102:2376/v1.16/containers/create: x509: certificate is valid for 0.0.0.0, not 192.168.99.102

Questions

What did go wrong?
Where should I check for the valid certificate?
Does it have to do with boot2docker?

thanks!

UPDATE 1:

I notice the command "ssh" and "upgrade"... So, I checked the Docker version inside the VirtualBox from machine, and it was 1.2.0...

$ machine ssh dev
                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
boot2docker: 1.2.0
             master : 8a06c1f - Fri Nov 28 17:03:52 UTC 2014
docker@boot2docker:~$ exit

So, I updated...

$ machine upgrade dev
INFO[0000] Stopping machine...                          
INFO[0006] Downloading boot2docker...                   
INFO[0012] Starting machine...                          
INFO[0012] Waiting for VM to start...      
$ machine ssh dev
                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.4.1, build master : 86f7ec8 - Tue Dec 16 23:11:29 UTC 2014
Docker version 1.4.1, build 5bc2ff8

However, the problem is that it even messed it up worse...

$ docker ps
FATA[0000] An error occurred trying to connect: Get https://192.168.99.102:2376/v1.16/containers/json: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "serial:125070080598599254051093126561600792625")

Document what images are used on what drivers

We should be clear about what operating systems are used on each provider (e.g. boot2docker on VirtualBox, Ubuntu on Digital Ocean and Azure).

Azure machines don't have identity auth

I'm using Mac OS X: https://bfirsh.s3.amazonaws.com/docker/darwin/docker-1.3.1-dev-identity-auth

./machine create -d azure --azure-publish-settings-file="MY-credentials.publishsettings" nodeone
INFO[0000] Creating Azure host...
INFO[0054] Waiting for SSH...
INFO[0242] Waiting for docker daemon on host to be available...
INFO[0304] "nodeone" has been created and is now the active machine. To point Docker at this machine, run: export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity
[644][hleclerc.machl: machine]$ export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity
[645][hleclerc.machl: machine]$ docker ps

2014/12/06 11:44:25 Get https://docker-host-b045d14b1d8e.cloudapp.net:4243/v1.15/containers/json: tls: oversized record received with length 20527

Use codegangsta/cli

The thing copied from Docker isn't very good.

Home directory should be detected in the same way as Docker

#65 (comment)

Display better error message when "ssh-keygen" is missing

Digitalocean driver assumes ssh-keygen is installed, if not it leaves a node in an error state. I hade to machine rm test and install ssh-keygen and recreate the node. A check for public key and/or ssh-keygen before creating the node might solve this.

FATA[0000] Host "test" already exists
root@4a6e561799c5:/go# machine ls
ERRO[0000] error getting state for host test: GET https://api.digitalocean.com/v2/droplets/0: 404 The resource you were accessing could not be found.
ERRO[0000] error getting URL for host test: IP address is not set
NAME   ACTIVE   DRIVER         STATE   URL
test            digitalocean   Error

Set the hostname of created machines to match the machine name

When creating and managing multiple machines, it is confusing to identify which machine you have logged into after using machine ssh. Setting the hostname to match the machine name would be useful in this scenario.

(woops I initially raised this against swarm by accident)

Windows : strange behaviour

Running the latest 0.0.2 in Windows results in this beahviour :

C:\Utilisateurs\a503140\Downloads>machine_windows_amd64.exe create -d virtualbox dev
←[31mFATA←[0m[0000] Identity auth public key does not exist at C:\Utilisateurs\a503140\.docker\public-key.json. Please run the docker client without any options to create it.

What seems strange is that code seems to be run inside the machine, and not outside (utils.go) :

cmdString := fmt.Sprintf("mkdir -p %q && cat > %q", authorizedKeysPath, filepath.Join(authorizedKeysPath, "docker-host.json"))
    cmd, err := d.GetSSHCommand(cmdString)

Configuration options though environment variables

It would be good if machine create used environment variables for some configuration options so we don't have to specify them every time. The DigitalOcean access token immediately comes to mind.

export DIGITALOCEAN_ACCESS_TOKEN=abcd123...

I think this might speed up development in some cases.

Add -v flag to report version

$ machine -v
0.0.1

x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error"

I have a new installation of Docker 1.4.1 and Boot2docker 1.4.1... Everything has a matching version but the Docker run command fails:

docker run busybox echo hello world
FATA[0000] An error occurred trying to connect: Post https://192.168.99.102:2376/v1.16/containers/create: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "serial:125070080598599254051093126561600792625")

Any suggestions?

Virtualbox vm does not start on osx Yosemite

./machine -D create -d virtualbox host1 wgillmer@not-used
INFO[0000] Downloading boot2docker...
INFO[0007] Creating SSH key...
DEBU[0007] executing: /usr/bin/ssh-keygen ssh-keygen -t rsa -N -f /Users/wgillmer/.docker/hosts/host1/id_rsa

Generating public/private rsa key pair.
Your identification has been saved in /Users/wgillmer/.docker/hosts/host1/id_rsa.
Your public key has been saved in /Users/wgillmer/.docker/hosts/host1/id_rsa.pub.
The key fingerprint is:
98:6c:22:95:3d:10:5d:58:03:05:09:62:49:8f:90:a6 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|.o+.++oO= |
|.+.+ ++ . |
|o . + o |
|E . . + |
| . . = S |
| . o |
| |
| |
| |
+-----------------+
INFO[0007] Creating VirtualBox VM...
DEBU[0007] Creating 20000 MB hard disk image...
Converting from raw image file="stdin" to file="/Users/wgillmer/.docker/hosts/host1/disk.vmdk"...
Creating dynamic image with size 20971520000 bytes (20000MB)...
DEBU[0011] executing: VBoxManage createvm --name docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --register
Virtual machine 'docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581' is created and registered.
UUID: ec142942-83dc-4583-8079-0b41859832b7
Settings file: '/Users/wgillmer/VirtualBox VMs/docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581/docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581.vbox'
DEBU[0011] executing: VBoxManage modifyvm docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --firmware bios --bioslogofadein off --bioslogofadeout off --natdnshostresolver1 on --bioslogodisplaytime 0 --biosbootmenu disabled --ostype Linux26_64 --cpus 8 --memory 1024 --acpi on --ioapic on --rtcuseutc on --cpuhotplug off --pae on --synthcpu off --hpet on --hwvirtex on --nestedpaging on --largepages on --vtxvpid on --accelerate3d off --boot1 dvd
DEBU[0011] executing: VBoxManage modifyvm docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --nic1 nat --nictype1 virtio --cableconnected1 on
DEBU[0011] executing: VBoxManage modifyvm docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --natpf1 ssh,tcp,127.0.0.1,53203,,22
DEBU[0011] executing: VBoxManage list hostonlyifs
DEBU[0012] executing: VBoxManage modifyvm docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --nic2 hostonly --nictype2 virtio --hostonlyadapter2 vboxnet2 --cableconnected2 on
DEBU[0012] executing: VBoxManage storagectl docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --name SATA --add sata --hostiocache on
DEBU[0012] executing: VBoxManage storageattach docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --storagectl SATA --port 0 --device 0 --type dvddrive --medium /Users/wgillmer/.docker/hosts/host1/boot2docker.iso
DEBU[0012] executing: VBoxManage storageattach docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --storagectl SATA --port 1 --device 0 --type hdd --medium /Users/wgillmer/.docker/hosts/host1/disk.vmdk
DEBU[0012] executing: VBoxManage guestproperty set docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 /VirtualBox/GuestAdd/SharedFolders/MountPrefix /
DEBU[0012] executing: VBoxManage guestproperty set docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 /VirtualBox/GuestAdd/SharedFolders/MountDir /
DEBU[0012] executing: VBoxManage sharedfolder add docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --name Users --hostpath /Users --automount
DEBU[0012] executing: VBoxManage setextradata docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 VBoxInternal2/SharedFoldersEnableSymlinksCreate/Users 1
INFO[0012] Starting VirtualBox VM...
DEBU[0012] executing: VBoxManage startvm docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581 --type headless
Waiting for VM "docker-host-8765264824c0107a9c55eca2b5094ab33616a646be97ab06888310cb4fbcc581" to power on...
VBoxManage: error: Locking of attached media failed
VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component SessionMachine, interface IMachine
INFO[0012] Waiting for VM to start...

Never get's further than that.

Automatically trust machine identity fingerprints

When you connect to a machine for the first time, it asks if you want to trust the machine you are connecting to. This isn't necessary – we can add the machine's fingerprint to known-hosts.json with SSH when setting up the machine.

docker run fails with "malformed HTTP response"

On OSX Yosemite with release 0.0.2:

$ machine -D create -d virtualbox cassandra
DEBU[0000] executing: VBoxManage

[Skipping VBoxManage output, hundreds of line of help]

INFO[0000] Downloading boot2docker...
INFO[0037] Creating SSH key...
DEBU[0037] executing: /usr/bin/ssh-keygen ssh-keygen -t rsa -N  -f /Users/nicolas/.docker/hosts/cassandra/id_rsa

Generating public/private rsa key pair.
Your identification has been saved in /Users/nicolas/.docker/hosts/cassandra/id_rsa.
Your public key has been saved in /Users/nicolas/.docker/hosts/cassandra/id_rsa.pub.
The key fingerprint is:
db:51:27:90:d7:8e:f4:68:35:80:94:cd:96:6c:ca:60 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|         .oB.+   |
|        E +.X +  |
|       . o *o*.. |
|          o.+oo  |
|        S ..     |
|         o .     |
|        . .      |
|                 |
|                 |
+-----------------+
INFO[0037] Creating VirtualBox VM...
DEBU[0037] Creating 20000 MB hard disk image...
Converting from raw image file="stdin" to file="/Users/nicolas/.docker/hosts/cassandra/disk.vmdk"...
Creating dynamic image with size 20971520000 bytes (20000MB)...
DEBU[0041] executing: VBoxManage createvm --name docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --register
Virtual machine 'docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78' is created and registered.
UUID: c724b220-7164-410c-be40-86a8d08d6fae
Settings file: '/Users/nicolas/VirtualBox VMs/docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78/docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78.vbox'
DEBU[0041] executing: VBoxManage modifyvm docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --firmware bios --bioslogofadein off --bioslogofadeout off --natdnshostresolver1 on --bioslogodisplaytime 0 --biosbootmenu disabled --ostype Linux26_64 --cpus 8 --memory 1024 --acpi on --ioapic on --rtcuseutc on --cpuhotplug off --pae on --synthcpu off --hpet on --hwvirtex on --nestedpaging on --largepages on --vtxvpid on --accelerate3d off --boot1 dvd
DEBU[0041] executing: VBoxManage modifyvm docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --nic1 nat --nictype1 virtio --cableconnected1 on
DEBU[0041] executing: VBoxManage modifyvm docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --natpf1 ssh,tcp,127.0.0.1,49261,,22
DEBU[0041] executing: VBoxManage list hostonlyifs
DEBU[0041] executing: VBoxManage hostonlyif create
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
DEBU[0041] executing: VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.99.1 --netmask 255.255.255.0
DEBU[0041] executing: VBoxManage list dhcpservers
DEBU[0042] executing: VBoxManage dhcpserver modify --netname HostInterfaceNetworking-vboxnet0 --ip 192.168.99.2 --netmask 255.255.255.0 --lowerip 192.168.99.100 --upperip 192.168.99.254 --enable
DEBU[0042] executing: VBoxManage modifyvm docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --nic2 hostonly --nictype2 virtio --hostonlyadapter2 vboxnet0 --cableconnected2 on
DEBU[0042] executing: VBoxManage storagectl docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --name SATA --add sata --hostiocache on
DEBU[0042] executing: VBoxManage storageattach docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --storagectl SATA --port 0 --device 0 --type dvddrive --medium /Users/nicolas/.docker/hosts/cassandra/boot2docker.iso
DEBU[0042] executing: VBoxManage storageattach docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --storagectl SATA --port 1 --device 0 --type hdd --medium /Users/nicolas/.docker/hosts/cassandra/disk.vmdk
DEBU[0042] executing: VBoxManage guestproperty set docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 /VirtualBox/GuestAdd/SharedFolders/MountPrefix /
DEBU[0042] executing: VBoxManage guestproperty set docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 /VirtualBox/GuestAdd/SharedFolders/MountDir /
DEBU[0042] executing: VBoxManage sharedfolder add docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --name Users --hostpath /Users --automount
DEBU[0042] executing: VBoxManage setextradata docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 VBoxInternal2/SharedFoldersEnableSymlinksCreate/Users 1
INFO[0042] Starting VirtualBox VM...
DEBU[0042] executing: VBoxManage startvm docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78 --type headless
Waiting for VM "docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78" to power on...
VM "docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78" has been successfully started.
INFO[0042] Waiting for VM to start...
DEBU[0073] Adding key to authorized-keys.d...
DEBU[0073] executing: ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 49261 -i /Users/nicolas/.docker/hosts/cassandra/id_rsa docker@localhost mkdir -p "/root/.docker/authorized-keys.d" && cat > "/root/.docker/authorized-keys.d/docker-host.json"
DEBU[0074] executing: ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 49261 -i /Users/nicolas/.docker/hosts/cassandra/id_rsa docker@localhost sudo /etc/init.d/docker restart
INFO[0074] "cassandra" has been created and is now the active machine. To point Docker at this machine, run: export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity

$ export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity

$ machine ls
NAME        ACTIVE   DRIVER       STATE     URL
cassandra   *        virtualbox   Running   tcp://192.168.99.100:2376

$ docker run busybox bash
2014/12/11 23:58:54 Post http://192.168.99.100:2376/v1.15/containers/create: malformed HTTP response "\x15\x03\x01\x00\x02\x02\x16"

$ docker run busybox echo hello world
2014/12/11 23:59:07 Post http://192.168.99.100:2376/v1.15/containers/create: malformed HTTP response "\x15\x03\x01\x00\x02\x02\x16"

$ ./docker-1.3.1-dev-identity-auth run busybox echo hello world
FATA[0000] TLS Handshake error: remote error: bad certificate

$ machine url
tcp://192.168.99.100:2376

$ echo $DOCKER_HOST
tcp://192.168.99.100:2376

$ echo $DOCKER_AUTH
identity

$ machine -D inspect cassandra
{
    "DriverName": "virtualbox",
    "Driver": {
        "MachineName": "docker-host-ae75949676e858b3896d54d65588cb4405837113ffe9025e80ffdeee9239aa78",
        "SSHPort": 49261,
        "Memory": 1024,
        "DiskSize": 20000,
        "Boot2DockerURL": ""
    }
}

$ curl -s http://192.168.99.100:2376/v1.15/ | xxd
0000000: 1503 0100 0202 0a                        .......

What am I missing?

Docker unable to connect to VM after restarting laptop.

Client version: 1.3.1-dev
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 831d09f
OS/Arch (client): darwin/amd64

I used docker machine successfully last night and turned of my laptop. This
morning I turned my laptop back on and ran the following commands:

machine start dev
machine active dev
DOCKER_HOST=$(machine url) DOCKER_AUTH=identity docker build --tag=target .

I however get the following error:

FATA[0000] TLS Handshake error: remote error: bad certificate

I can fix this:

machine stop dev
machine rm dev
machine create -d virtualbox dev

However I lose all the cached layers in my previous VM.

Shared folders should be configurable for local VM drivers

Add an option to make `machine ls` output machine readable

It would be great have something like

$ machine ls --format json
[
    {
        "name": "dock001",
        "active": true,
        "driver": "virtualbox",
        "state": "running",
        "url": "tcp://192.168.0.1:2376"
    },
    {
        "name": "dock002",
        "active": false,
        "driver": "openstack",
        "state": "running",
        "url": "tcp://192.168.0.2:2376"
    }
]

Machines are in a broken state when creation fails

Consider this example:

$ machine ls
NAME   ACTIVE   DRIVER   STATE   URL

$ machine create -d amazonec2 --amazonec2-access-key foo --amazonec2-secret-key bar --amazonec2-subnet-id quux spam
INFO[0000] Launching instance...
FATA[0000] Problem with AWS API call: Non-200 API response: code=401 message=AWS was not able to validate the provided access credentials

$ machine ls
ERRO[0000] error getting state for host spam: Problem with AWS API call: Non-200 API response: code=401 message=AWS was not able to validate the provided access credentials

ERRO[0000] error getting URL for host spam: Problem with AWS API call: Non-200 API response: code=401 message=AWS was not able to validate the provided access credentials

NAME   ACTIVE   DRIVER      STATE   URL
spam            amazonec2   Error

$ machine rm spam
ERRO[0000] Error removing machine spam: unabme to terminate instance: unknown instance
FATA[0000] There was an error removing a machine. To force remove it, pass the -f option. Warning: this might leave it running on the provider.

$ machine ls
ERRO[0000] error getting state for host spam: Problem with AWS API call: Non-200 API response: code=401 message=AWS was not able to validate the provided access credentials

ERRO[0000] error getting URL for host spam: Problem with AWS API call: Non-200 API response: code=401 message=AWS was not able to validate the provided access credentials

NAME   ACTIVE   DRIVER      STATE   URL
spam            amazonec2   Error

$ machine rm -f spam
ERRO[0000] Error removing host, force removing anyway: unabme to terminate instance: unknown instance

$ machine ls
NAME   ACTIVE   DRIVER   STATE   URL

User should not have to machine rm -f at all. If there is any error with machine create, machine should clean up the failed host create.

Make boot2docker compatible with Machine

I'm using this issue as a way of tracking our boot2docker dependencies for a Machine 1.0 release.

Docker with identity auth should be included in boot2docker
This patch is required for VMware Fusion

[Feature Request] ls should report daemon availability

Currently the ls command reports whether a host is active, but it does not report daemon availability. machine ls should check using something like net.DialTimeout to see if the daemon is responding to requests as well as if the host is up.

Add homebrew formulas

I knocked up a quick Formula to install machine already. Perhaps you can link to that somewhere in your docs for people running OS X.

Cache local base VM images

When creating multiple machines locally the base VM image is downloaded multiple times. Creating machines would be a nicer experience if the images were cached. This would allow multiple machines to be spun up consecutively:

rob@discovery:~$ machine create -d virtualbox swarm0
INFO[0000] Downloading boot2docker...                   
INFO[0035] Creating SSH key...                          
INFO[0035] Creating VirtualBox VM...                    
INFO[0041] Starting VirtualBox VM...                    
INFO[0041] Waiting for VM to start...                   
INFO[0072] "swarm0" has been created and is now the active machine. Docker commands will now run against that machine. 
rob@discovery:~$ machine create -d virtualbox swarm1
INFO[0000] Downloading boot2docker...

machine url command for virtualbox fails on Windows

First issue: In virtualbox.go GetState uses a regex to pull the value out of "VMState". The regex is:

re := regexp.MustCompile(`(?m)^VMState="(\w+)"$`)

My wild guess on the first try was that Windows is different because of \r\n. Whether that was it or not, removing the $ to match end-of-line fixed the problem.

Second issue: driver still can't get the ip address.
This is done in the GetIP func in virtualbox.go. Quick debugging, the stuff that parses the results from "ip addr show dev eht1" can't find "inet". On my machine it looks like there is an ipv6 addr in the result. Haven't compared on a Mac or Linux machine to see what is there.

[Proposition] Make optional machine name for some commands

Some CLI commands like ssh, start, stop... should be executed on the active machine if no machine is specified in the command.

For instance, here

$ machine ls

NAME      ACTIVE   DRIVER      STATE     URL
dock001            openstack   Running   tcp://192.168.0.152:2376
dock002   *        openstack   Running   tcp://192.168.0.153:2376
dock003            openstack   Running   tcp://192.168.0.154:2376

commands machine ssh and machine stop should be equal to machine ssh dock002 and machine stop dock002.

Dockerfile to run Machine

Hi, I see there is already a Dockerfile for building Machine, but I created a Dockerfile to run Machine inside a container using docker-in-docker. Wasn't sure if there was a place for it in this repo, or if I should just keep it as a separate repo and perhaps reference it in the docs for those interested? I'm also fine if this issue is closed and the runtime Dockerfile doesn't belong here, but thought I'd throw it out there if there's interest.

On the hub:
https://registry.hub.docker.com/u/defermat/machine/

On Github:
https://github.com/defermat/machine

I'm happy to make a PR if there's a particular direction that makes sense.

Export / import machines

Something like this should work for transferring machines:

 $ machine export test | ssh anotherhost machine import

(Maybe we could use machine inspect?)

Make VirtualBox port forwarding configurable

Hi,

It would be helpful if we can configure the port forwarding when installing a new box.

Thanks

add auth command

It would be nice to have a machine auth -d <driver> command that authorize machine with a given provider's driver.

That way each driver could cache and manage credentials, and there wouldn't be any additional authorization step as part of machine create

[Feature Request] Configure Docker client inside machine

After creating a new machine, the Docker client on the created machine itself is not configured to connect to the daemon. It would be nice if the client on the machine is also automatically configured.

# Local client is able to connect to the remote daemon;
./docker-1.3.1-dev-identity-auth info
[outputs docker info of "machine"]

# But SSH-ing into the remote machine, that client is not configured

./machine ssh dev

~# docker info
FATA[0000] Cannot connect to the Docker daemon. Is 'docker -d' running on this host? 

~#docker --host=tcp://127.0.0.1:2376 --auth=identity info
FATA[0000] TLS Handshake error: remote error: bad certificate

[Feature Request] REST(ish) API

docker machine is evolving towards becoming a useful tool for programmers and systems administrators. It is likely that a demand for:

higher-order functionality such as coordinating the complex interactions and topologies of an Amazon VPC
ways to automate docker machine actions via programming language of choice without shelling out

will emerge in the future as the base gets more stable and robust. I would propose that machine should allow the ability to run in a server mode which accepts API requests (or, we provide some way of wrapping machine which does). It would look something like:

To start server:

$ machine -d -p 7000 
Listening for machine requests on 127.0.0.1:7000....

[in a separate window]

$ curl --silent \
    -X POST \
    -H 'Content-Type: application/json' \
    http://localhost:7000/create \
    -d '{"Driver": "ec2", "AwsAccessKeyId": "blah", "AwsSecretKey": "foo", "HostName": "my-amazon"}'
{
   "HostName":"my-amazon",
   "Status":"Starting"
}

API responds rapidly to request for creating machine and queues action in background. Then user is freed up to do more stuff e.g. check status.

$ curl --silent \
    -X GET \
    http://localhost:7000/ls
[
   {
      "Name":"default",
      "Active":true,
      "Driver":"",
      "State":"",
      "Url":"unix:///var/run/docker.sock"
   },
   {
      "Name":"my-amazon",
      "Active":false,
      "Driver":"ec2",
      "State":"starting",
      "Url":"tcp://50.116.43.32:2376"
   }
]

It allows the possibility of long-running concurrent actions that aren't possible with the current (non-backgrounded) implementation as well- e.g. create a host on Digital Ocean, Rackspace, and Azure concurrently. Right now users would have to "bring their own concurrency" using & or some other solution.

Machine should configure Docker to automatically control the active machine

We should configure the Docker host and authentication method so this works seamlessly:

$ machine create -d virtualbox dev
$ docker run ...

The best way to do this is probably to create a configuration file that sets these options (~/.docker/config?). This will require modification to Docker – I'll put together a proposal on the Docker repository.

An alternative method could be to set environment variables. For example, there could be a machine config command which output:

export DOCKER_HOST=$(machine url) DOCKER_AUTH=identity`

You could then just run $(machine config) and your shell would be set up to talk to the active machine.

Repository compromised?

The wiki for this repo : https://github.com/docker/machine/wiki/ seems to have a lot of Indonesian spam content. Has this repository been compromised?
CC @bfirsh @ehazlett

Remove VirtualBox NAT network when it is no longer required

boot2docker-cli checks if any machines are using the VirtualBox NAT network and removes it when it is not needed. We should do this too so we aren't polluting the VirtualBox host with unnecessary networking interfaces.