Currently there's code like this in Kaiserfiles.

app_params " -e CHROME_HOSTNAME=<%= @config[:shared_names][:chrome] %>"

That @config variable is code that gets eval'd in an instance of Kaiser::Cli and thus you need to know the internals of that class in order to write Kaiserfiles and refactoring it becomes hard without breaking existing Kaiserfiles.

When there is time, it would be a good idea to think of a way to make this more abstract.

Currently there aren't really any example of how to write Kaiserfiles yet. For us internally it's probably not that much of a problem since we can look at some projects already using Kaiser.

For people outside of Degica to use it, it would be very good to have some examples and/or documentation on how to write a Kaiserfile for their app.

Context

When running kaiser up in my project, the app hasn't come up but kaiser didn't show me an error message and exited with status 0. Because of this it took me some time to figure out that an error had occurred at all.

Expected behaviour

Print an error message to stdout and exit with a non zero status.

Actual behaviour

Shows output like the following and exits with status 0

Starting up database                                                                                                                                                   
Starting up ned-db                                                                                                                                                     
.....                                                                                                                                                                  
Waiting for database to start...                                                                                                                                       
................                                                                   
Started.                                                                                                                                                               
Starting up application                                                                                                                                                
.........

Image name is treated as platform:

Starting up database
Starting up my-db
> docker run -d
        -p 9004:5432
        -v my-database:/var/lib/postgresql/data
        --name monarch-db
        --network kaiser_net
        -e POSTGRES_PASSWORD=example
        --platform  postgres:alpine

"docker run" requires at least 1 argument.
See 'docker run --help'.

Usage:  docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

I think we should add a note about running bundle install prior to building the kaiser container. Gemfile.lock won't exist otherwise, and the build will error out:

~/Documents/kaiser • docker build -t degica/kaiser .
Sending build context to Docker daemon  547.3kB
Step 1/10 : FROM ruby:alpine
 ---> b46ea0bc5984
Step 2/10 : RUN apk update && apk add docker build-base git curl
 ---> Using cache
 ---> 23ba2c3df671
Step 3/10 : ADD bin /app/bin
 ---> Using cache
 ---> ddc67914747e
Step 4/10 : ADD exe /app/exe
 ---> Using cache
 ---> 810a45cef209
Step 5/10 : ADD lib /app/lib
 ---> Using cache
 ---> 61d8bf69554f
Step 6/10 : ADD spec /app/spec
 ---> Using cache
 ---> 3590e2e55265
Step 7/10 : ADD Gemfile Gemfile.lock kaiser.gemspec Rakefile entrypoint.sh /app/
ADD failed: stat /var/lib/docker/tmp/docker-builder177632162/Gemfile.lock: no such file or directory

The app user often ends up with a UID that's different from the host UID. When you attach, you'll find that all shared files have the host's UID. This causes permission errors when trying to do some things.

Context

There's quite a bit of duplication across some of our Kaiser projects which check that submodules exist. For example, HATS and HCS both define the following in their kaiser files:

`git submodule status`.lines.each do |line|
  # The git-submodule man page says uninitialized submodules are prefixed with a -
  # but I found this unreliable. While testing I pressed Control-C in the middle of
  # the update command so some submodule would be initialized and others wouldn't.
  # After that, the status command had removed the - for every submodule.
  # Therefore we just check if there's files in the directory instead.
  dir = line.strip.split(' ')[1]
  if Dir.empty?(dir)
    puts "Found uninitialized git submodule '#{dir}'"
    puts "please run 'git submodule update --init --recursive'"
    exit 1
  end
end

Of course this would tie Kaiser to git but that's maybe not a bad thing. If there is a use-case without git than maybe this could be implemented as a helper method e.g. check_git_submodules!

When a different docker environment is running and is bound to the same ports that Kaiser is trying to use when it starts, it logs a message and fails to bind the ports for that particular container. However, it then continues with the rest of the setup. This can result in confusing error messages as later steps in the process are assuming that the container was bound successfully.

A possible improvement for this would be for kaiser to fail if the container cannot be bound, and notify the user that the required port is in use and that they need to stop whatever is using that port and retry kaiser again.

Example of how kaiser handles attempting to bind to a port already in use. The error message doesn't really stand out and can easily get buried in the rest of the output text:

docker: Error response from daemon: driver failed programming external connectivity on endpoint nginx (f561316244b4db5830ff146df9b50b25b444ed7aff158081595e895663ec0476): Bind for 0.0.0.0:443 failed: port is already allocated.

# continues on with the rest of the process...

How to replicate

1. Start a separate docker container and bind it to the same port as one that Kaiser will try to use
2. start Kaiser
3. Note a similar error message as the one above and that kaiser continues with the rest of the process

Right now when you kaiser up for the first time it will display some errors. These errors are thrown to stderr by the cli which kaiser uses.

These errors make it seem like theres something wrong when there isn't, which makes the user suspicious and terminate when he feels like it.

Currently there aren't any real tests for Kaiser yet. Maybe provide a simple demo Kaiserfile for which we can test all features?

We should make a plugin system where plugins can be pulled in by adding gems into a gemfile, and then we would solve #45 #38 #27 #23 by writing plugins for it

Current kaiserfiles have huge amounts of code to define how to database:

db 'postgres:alpine',
   port: 5432,
   data_dir: '/var/lib/postgresql/data',
   params: '-e POSTGRES_PASSWORD=example',
   waitscript_params: "
     -e PG_HOST=<%= db_container_name %>
     -e PG_USER=postgres
     -e PGPASSWORD=example
     -e PG_DATABASE=postgres",
   waitscript: <<~SCRIPT
     #!/bin/sh

     RETRIES=5

     until psql -h $PG_HOST -U $PG_USER -d $PG_DATABASE -c "select 1" > /dev/null 2>&1 || [ $RETRIES -eq 0 ]; do
       echo "Waiting for postgres server, $((RETRIES--)) remaining attempts..."
       sleep 1
     done
   SCRIPT

This is nice and general and suitable for a large family of databases that are in practical use today.

But need to make this easier for well-known databases, or make something like a Kaiser plugin repository where kaiser can simply retrieve them.

Something like

database :postgres

And if you want to specify an image

database :postgres, image: 'postgres:alpine'

If you want to add additional params (ala KOMOJU)

database :mysql, image: 'mysql:5.6', commands: <<~CMD
    --character-set-server=utf8mb4
    --collation-server=utf8mb4_unicode_ci
    --innodb-large-prefix
    --innodb-file-format=barracuda
CMD

And maybe some well known params for the database:

database :mysql, env: {
  MYSQL_ROOT_PASSWORD: 'qwerty'
}

Display better errors when a user terminates kaiser prematurely instead of just showing the ruby backtrace

Display a proper error message if the user does not have docker or dockerd is not active.

Context

When attaching to an image that doesn't exist Kaiser should return an error message
saying something like:

kaiser:hats-master image could not be found.
Please run "kaiser up" in your current directory to build image.

Context

I noticed the -v flag needs to be added before the command in order to work. We should improve handling these options better so they can be added at the end.

Works

kaisuer -v up

Doesn't Work

kaiser up -v

Right now kaiser is hardcoded to share chrome and redis amongst all its apps. It would be much better if the apps themselves asked for the shared services they want, so starting up Kaiser that does not require anything would not start them, or apps can choose to have both chrome and firefox test containers, say.

I think the most natural expectation for a command named kaiser login is to get a shell on a container. It however just executes a command and to get the shell you have to run kaiser login sh.

I believe it's better to split this into two commands:

• kaiser exec to run commands.
• kaiser login to get a shell. This would be equivalent to running kaiser exec sh

While we're at it, we might want to add a parameter to specify which container we want to do this against. Currently it's hard coded to the app container and I this should remain the default but perhaps a parameter like the following is a good idea?

kaiser exec uptime #runs uptime on the app container
kaiser exec -c db uptime #runs uptime on the db container

kaiser login # gets us a shell on the app container
kaiser login -c db #gets us a shell on the db container

Context

One of the pain points for me when using Kaiser is the time it takes to download SSL certifications. This should be optimized or disabled by default as SSL needs to be tested very rarely in development mode.

Context

Kaiser assumes a database is required and has commands like db_reset_command and db

Some projects don't require a database. Kaiser currently throws an exception when no database is defined in a Kaiserfile.

(output when running kaiser up -v)

> docker volume create docs-database
docs-database
$? = 0
Starting up database
Traceback (most recent call last):
        7: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/bin/kaiser:23:in `<main>'
        6: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/bin/kaiser:23:in `load'
        5: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/exe/kaiser:81:in `<top (required)>'
        4: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/lib/kaiser/cli.rb:75:in `run_command'
        3: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/lib/kaiser/cmds/up.rb:21:in `execute'
        2: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/lib/kaiser/cli.rb:113:in `setup_db'
        1: from /Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/lib/kaiser/cli.rb:183:in `start_db'
/Users/richardramsden/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/kaiser-0.4.5/lib/kaiser/cli.rb:359:in `db_expose': undefined method `[]' for nil:NilClass (NoMethodError)

Right now docker just complains the image does not exist. Attach should build the image if it does not exist.

This seems to be a very common thing in our Kaiserfiles:

github_webhook_secret = ENV['GITHUB_WEBHOOK_SECRET'] || 'aaaaaa'
twilio_webhook_secret = ENV['TWILIO_WEBHOOK_SECRET'] || 'abcdef'
twilio_webhook_url = ENV['TWILIO_WEBHOOK_URL'] || 'https://dff80da3.ngrok.io/api/v1/webhook/twilio/sms'
ping_channel = ENV['GITHUB_PING_CHANNEL'] || 'dsiawlab'

app_params "
  -e RAILS_ENV='#{rails_env}'
  -e GITHUB_ORGANIZATION=#{github_org}
  -e GITHUB_TOKEN=#{github_token}
  -e SLACK_TOKEN=#{slack_token}
  -e GITHUB_WEBHOOK_SECRET=#{github_webhook_secret}
  -e TWILIO_WEBHOOK_SECRET=#{twilio_webhook_secret}
  -e TWILIO_WEBHOOK_URL=#{twilio_webhook_url}

We can possibly start abstracting it into a new syntax for kaiserfiles such as:

input_var :GITHUB_WEBHOOK_SECRET, default: 'aaaaaa'

Also ,typically we read Kaiser values like this:

app_params "
  -e CHROME_HOSTNAME=<%= @config[:shared_names][:chrome] %>

Also providing values can be done like this:

input_var :CHROME_HOSTNAME, default: ->{ @config[:shared_names][:chrome] }

This will add additional params to our docker run command which helps make writing Kaiserfiles easier.

Context

I noticed hats-tester.localhost.labs.degica.com redirects to a random live Kaiser instance if the project is not actually running.

This caused some confusion when I was trying to run both projects.

I'm currently writing some documentation for Kaiser and while digging into the tool's internals I missed the ~/.kaiser/.config.yml for quite a while.

Since ~/.kaiser is already a hidden directory, I think the files in there shouldn't be hidden. I suggest changing it to ~/.kaiser/config.yml and building in a little backwards compatibility layer for people that still have the old .config.yml I'd suggest a function that looks for the dotted file and undots if it exists. If an undotted config file exists already, it should take priority over the dotted one.

When kaiser is working, it is not clear if it has crashed or is running.

Perhaps we need to display dots as as a sign that it is busy