Code Monkey home page Code Monkey logo

pool's Introduction

🐳 pool 🐳

wercker status

The simplest proxy service to access your Dockerized web application by Git commit-id, branch or tag.

You can build and run your web application as a Docker container just to access http://<git-commit-id, branch or tag>.pool.dev for example.


Requirements

  • Docker should be installed on your host machine
    • Our Vagrantfile is configured to have docker so you don't have to care about it if you try to run on Vagrant
  • Web application which you want to preview should be on Git repository
  • Git repository of the application has Dockerfile on root of the repository
    • Dockerfile is for a container which the application is going to be run

Quick start

Vagrant

You can run pool on your local Vagrant environment also.

Launch Vagrant box

It needs to install vagrant dns plugin before vagrant up. Just run after vagrant was installed:

vagrant plugin install vagrant-dns
vagrant dns --install
vagrant dns --start

then run:

vagrant up

Parameters

You can give some configuration to pool in Vagrantfile.

Currently we some configurations for pool.

  • github-bot
    • It enables github integration
  • Git repository url
    • Your Git repository URL
  • Maximum numbers of comtainers
    • pool kills containers if the number of containers are over than this number automatically
  • Hostname
    • Hostname you want to use to access the environment

You can rewrite Vagrantfile like:

# If you'd like to enable github integration, uncomment below
s.args << "--github-bot"

# Set your repository for previewing by pool
s.args << "https://github.com/mookjp/flaskapp.git"
# Set the maximum number of containers runnning at the same time
s.args << "5"
# Set POOL_BASE_DOMAIN
s.args << [pool_hostname, pool_tld].join(".")

Access web application

To watch the web application on pool, you can do it just to access http://<git-commit-id>.pool.dev with your browser.

In the default setting, pool is configured for the web application maintained in the mookjp/flaskapp repository. You can see the flask app (which just outputs 'hello world') just visiting http://c8f48c60088bbae0d0fb25ed5fd04f4442b58617.pool.dev/ or http://master.pool.dev/.

AWS EC2 instances

Following is an example to install pool to Amazon Linux.

Instalation for Amazon linux

To install pool on your Amazon Linux, use following userdata. This example is for Amazon Linux AMI amzn-ami-hvm-2014.03.2.x86_64-ebs (ami-29dc9228)

#!/bin/sh
# Setup script for pool
# NOTE: Run it as root
yum install -y git
yum install -y docker

# Install latest Docker
service docker stop
curl https://get.docker.com/builds/Linux/x86_64/docker-latest -o /tmp/docker
chmod +x /tmp/docker
cp /tmp/docker /usr/bin/docker
service docker start

# Download pool then run pool container
git clone https://github.com/mookjp/pool.git /app

# You can create `pool` image and run to execute `init_host_server` script.
# It gets 3 parameters:
# 1) Git repository URL
# 2) Maximum number of containers of web application
# 3) Hostname
/app/scripts/init_host_server "https://github.com/mookjp/flaskapp.git" 5 "dev.prevs.io"

Parameters of init-script

pool/scripts/init_host_server is a small util script to run pool container. It gets 3 parameters:

  1. Git repository URL
  2. Maximum number of containers of web application
  3. Hostname. you can get your hostname from your container of application as environment variable; POOL_BASE_DOMAIN so that you can set configuration related to hostname inside your container

Passing a SSH key to pool container

You may want to upload a SSH private key to pool container in case your git repository is hidden from public access.

Then you can place id_rsa at /app/docker/pool/keys/id_rsa in the server where pool is installed.

And id_rsa will be uploaded to /app/keys/id_rsa on the pool container which is going to be used to clone your application repository.

core@pool ~ $ cat<<'EOS'>/app/docker/pool/keys/id_rsa
> -----BEGIN RSA PRIVATE KEY-----
> ..........
> -----END RSA PRIVATE KEY-----
> EOS

core@pool ~ $ sudo /app/scripts/init_host_server

core@pool ~ $ sudo docker exec -it pool bash

[pool container]# ls -l /app/keys/id_rsa

How it works

The part of proxy in pool accesses your Git repository with commit id given as hostname then checkout source with Dockerfile. Dockerfile should be on the root of the repository. After checkout files, the container will be built by the Dockerfile and port is linked with front automatically. All you can do is just to access by URL like http://<git-commit-id, branch or tag>.pool.dev.

pool consists of two modules; proxy hook and container builder. handlers/hook.rb handles HTTP request as proxy. This is a hook script of matsumoto-r/mod_mruby. It forwards port which Docker container was assigned by Git-commit-id.

If there's no container which corresponds to Git-commit-id, build_server works to build Docker image then runs it. build_server sends build log so that you can confirm the status of build process while waiting.

If there is another proccess to build and run container, pool locks to run other process and waits until lock is over.

Contributors

Contributors

pool's People

Contributors

mookjp avatar ainoya avatar matsumotory avatar tomykaira avatar takahi-i avatar kenchan avatar blp1526 avatar

Stargazers

Yoshiki Nakagawa avatar  avatar ytakhs avatar Hidenori Maehara avatar Fumiaki MATSUSHIMA avatar  avatar aminalamin avatar Tsukasa Arima avatar Jun-ichiro Suzuki avatar Armel Soro avatar kroton avatar J. Felix Etcetera avatar bells17 avatar Vladimir Ivanov avatar Vladimir Ivanov avatar Yuki Wakayama avatar Anton Yurchenko avatar yoshinorin avatar Kay Yan avatar MB avatar Atsushi Yoshinari avatar gentaro avatar Chris Olstrom avatar Alexander Ivanovsky avatar haya14busa avatar misoobu avatar Masaki ISHIYAMA avatar Hiroki Sato avatar MOGI Hiromu avatar Naotoshi Seo avatar Yusuke Abe avatar Masaya Kamakura avatar Chris Hoffman avatar Alexandre Chaussier avatar yowatari avatar roadlabs avatar Dreamsome avatar Krzysztof Wilczyński avatar Takashi Fujita avatar Drew Nall avatar Beni Cherniavsky-Paskin avatar Mihyaeru avatar onigra avatar wiro34 avatar ongaeshi avatar bonty avatar Yasuaki Goto avatar Galmido avatar Masato Nakamura avatar kyokutyo avatar Oleksandr Simonov avatar Takuya Wakisaka avatar Shunsuke Wada avatar Yuji Takaesu avatar wshino avatar tady (GitHub) avatar  avatar Yasuharu Sawada avatar Taro MORIYA avatar Uğur Devril avatar  avatar Ryota Mochizuki avatar Yoshiya Hinosawa avatar Sho avatar Dezső Zoltán avatar  avatar  avatar takafumi tsuchida avatar Kouhei Maeda avatar Kenta Mori avatar Shuhei KONDO avatar Sho Kawakami avatar Shu Uesugi avatar Minoru KAWAMOTO avatar Ken Takagiwa avatar Reo Yoshida avatar  avatar Hiroki OHTSUKA avatar shikakun avatar Masato OSHIMA avatar Ethan Turkeltaub avatar Alexander ADAM avatar Tsuyoshi Torii avatar Huy Nguyen Quang avatar Tam Nguyen avatar Han Ngo avatar C avatar Man Vuong avatar Tomohiro Hashidate avatar Takeshi Yabe avatar Mitsutaka Mimura avatar flada-auxv avatar Kazuma Furuhashi avatar Hideaki Tanabe avatar Kenichi Tsunokawa avatar Shinji Yamada avatar John D'Agostino avatar Yuki Yoshinoya avatar oinume avatar Koichi Shiraishi avatar

Watchers

 avatar  avatar roadlabs avatar  avatar Mikoto Okumura avatar Sho Kawakami avatar  avatar Itsuki Sakitsu avatar tkmknr avatar wercker avatar gentaro avatar Koichi Shiraishi avatar  avatar

pool's Issues

Couldn't work with git repository name contains "-"

It's because of constraint of docker name.

console$ docker build -t aaaa-a/w_p-1 .
2014/10/18 20:02:39 Invalid namespace name (aaaa-a), only [a-z0-9_] are allowed, size between 4 and 30

I planned to add process to replace hyphen to underscore of repository name inside pool system.
But this fix causes new problem, pool cannot distinguish repository name app-abc or app_abc.
If there are more better solution, we should prevent this problem.

Enhance box building

Currently it takes so long time to execute provisioning to Vagrant box.

Plans

  1. Divide the task to create environment to another project; create pool's own Vagrant box project and make it as .box file
  2. Make rpm package
    • Ruby

Logo

I hope some kind of cute logo.

hook.rb makes defunct child process

root        32  0.0  0.7 190396  7996 ?        Ss   00:55   0:00 /usr/sbin/httpd
apache      35  0.0  0.6 190500  6580 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     110  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     144  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      37  0.0  0.6 190508  6584 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     172  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     181  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      38  0.0  0.6 190508  6716 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache      59  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      66  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      78  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      86  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     305  0.0  0.0      0     0 ?        Z    00:56   0:00  |   \_ [docker] <defunct>
apache     312  0.0  0.0      0     0 ?        Z    00:56   0:00  |   \_ [docker] <defunct>
apache     319  0.0  0.0      0     0 ?        Z    00:56   0:00  |   \_ [docker] <defunct>
apache     326  0.0  0.0      0     0 ?        Z    00:56   0:00  |   \_ [docker] <defunct>
apache      39  0.0  0.6 190500  6580 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache      94  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     102  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      40  0.0  0.6 190500  6580 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     112  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     145  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      41  0.0  0.6 190500  6580 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     111  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     140  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      42  0.0  0.6 190500  6264 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     207  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache     230  0.0  0.0      0     0 ?        Z    00:55   0:00  |   \_ [docker] <defunct>
apache      43  0.0  0.6 190500  6264 ?        S    00:55   0:00  \_ /usr/sbin/httpd
apache     208  0.0  0.0      0     0 ?        Z    00:55   0:00      \_ [docker] <defunct>
apache     228  0.0  0.0      0     0 ?        Z    00:55   0:00      \_ [docker] <defunct>

warning output while building the builder gem

When building docker,

Step 22 : RUN /usr/local/bin/gem build builder.gemspec
 ---> Running in 2998ac0631a8
WARNING:  description and summary are identical
WARNING:  open-ended dependency on em-websocket (>= 0) is not recommended
  if em-websocket is semantically versioned, use:
    add_runtime_dependency 'em-websocket', '~> 0'

Multiple application support

For cat:

  • http://master.cat-app.pool
  • http://development.cat-app.pool

For dog:

  • http://master.dog-app.pool
  • http://4fsocxnfd843d.dog-app.pool

Show error when accessing the branch doesn't exist

When accessing the branch name domain doesn't exist, error occurs inside builder as follows:


/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:764:in `command': git rev-parse 'development'  2>&1:fatal: ambiguous argument 'development': unknown revision or path not in the working tree. (Git::GitExecuteError)
Use '--' to separate paths from revisions
development
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:108:in `revparse'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/base.rb:541:in `revparse'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:19:in `block (3 levels) in <top (required)>'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/connection.rb:24:in `call'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/connection.rb:24:in `trigger_on_open'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/connection.rb:114:in `block in dispatch'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/em/deferrable.rb:151:in `call'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/em/deferrable.rb:151:in `set_deferred_status'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/em/deferrable.rb:191:in `succeed'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/handshake.rb:150:in `process'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/handshake.rb:29:in `receive_data'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/connection.rb:127:in `dispatch'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/em-websocket-0.5.1/lib/em-websocket/connection.rb:78:in `receive_data'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'
        from /opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:13:in `<top (required)>'
        from /opt/ruby-2.1.2/bin/build_server:23:in `load'
        from /opt/ruby-2.1.2/bin/build_server:23:in `<main>'
Connection closed

Interface to redirect to path

For now, if pool has no container of git-commit-id, the user should access to root of pool-server like http://master.pool.dev/. The user has to go to root to create new container. It means that the user would not see what his/her want to see like http://master.pool.dev/show/something.

This problem came from the way to serve build-screen in hook.rb.

Lock to build new container which is linked same git-commit-id at the same time

When some requests access to pool and builder run checkout in git or ADD in Docker at the same, it does not work.

I, [2014-12-05T17:26:33.196787 #62]  INFO -- : git config '--list'  2>&1

D, [2014-12-05T17:26:33.196847 #62] DEBUG -- : core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
remote.origin.url=http://fan_ope:[email protected]/fan/fan.git
branch.master.remote=origin
branch.master.merge=refs/heads/master

I, [2014-12-05T17:26:33.199490 #62]  INFO -- : git rev-parse 'remotes/origin/pool3'  2>&1

D, [2014-12-05T17:26:33.199571 #62] DEBUG -- : 60fd40ffcd5240a5dbc975e97ab37296c0a94627

I, [2014-12-05T17:26:33.199637 #62]  INFO -- : Build for 60fd40ffcd5240a5dbc975e97ab37296c0a94627 ...

I, [2014-12-05T17:26:33.581966 #62]  INFO -- : git checkout '60fd40ffcd5240a5dbc975e97ab37296c0a94627'  2>&1

D, [2014-12-05T17:26:33.582076 #62] DEBUG -- : error: Entry 'fan-frontend/src/index.hbs' would be overwritten by merge. Cannot merge.

E, [2014-12-05T17:26:33.582134 #62] ERROR -- : git checkout '60fd40ffcd5240a5dbc975e97ab37296c0a94627'  2>&1:error: Entry 'fan-frontend/src/index.hbs' would be overwritten by merge. Cannot merge. (Git::GitExecuteError)
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:764:in `command'
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:528:in `checkout'
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/base.rb:325:in `checkout'
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:203:in `build'
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:127:in `up'
/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:16:in `block (4 levels) in 
Git::GitExecuteError: git checkout '60fd40ffcd5240a5dbc975e97ab37296c0a94627'  2>&1:error: Entry 'fan-frontend/src/index.hbs' would be overwritten by merge. Cannot merge.; ["/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:764:in `command'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/lib.rb:528:in `checkout'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/git-1.2.8/lib/git/base.rb:325:in `checkout'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:203:in `build'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:127:in `up'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:16:in `block (4 levels) in

then reload:

pool$ build
Errno::EROFS: Read-only file system @ rb_sysopen - /app/images/builder.log; ["/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder/builder_log_device.rb:13:in `initialize'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder/builder_log_device.rb:13:in `new'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder/builder_log_device.rb:13:in `initialize'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:70:in `new'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/lib/builder.rb:70:in `initialize'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:15:in `new'", "/opt/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/builder-0.0.1/bin/build_server:15:in `block (4 levels) in

Add test

Currently this prototype is buggy. Should add tests.

docker environment variable "POOL_HOST_NAME"

In wordpress example, in order to replace hard-coded hostname inside mysql database, I used variable POOL_HOST_NAME.

See start.sh.

POOL_HOST_NAME is passed by docker run -e POOL_HOST_NAME="theme1.pool.dev" command.

In any other case, we probably need hostname which subdomain in pool is.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.