Code Monkey home page Code Monkey logo

mba's Introduction

Overview

Metabase Assembler is a development and repro tool based on docker-compose.

Dependencies

  • docker & docker-compose
  • babashka 2.12 or higher.

Install

  • install babashka. Download
  • install docker && docker-compose
  • git clone https://github.com/metabase/mba
  • If you want it in your path: ln -s ./mba/src/main.clj ~/bin/mba
  • mba --help or ./mba/src/main.clj --help

BugBash Instructions

If you’re bugbashing:

  • mba --mb dockerhub:metabase/metabase-head pull. Will pull latest metabase-head image.
  • mba --mb dockerhub:metabase/metabase-head up. Will spin metabase linked to a postgresql and maildev. Open `localhost:3000` and `localhost:1080`.
  • When you’re done, mba --mb dockerhub:metabase/metabase-head down. containers and data will be wiped.
  • Emergency hatch: mba nuke will kill everything mba related.

Usage & Commands

  • mba up . creates the stack
  • mba pull . pulls images from the stack. (useful if you use *-head)
  • mba shell. gives a shell inside the main metabase container
  • mba dbconsole. console into the app-db
  • mba run ls. runs one-off commands in the mb container
  • mba logs. show logs
  • mba compose logs -- --tail=100 -f. same, using raw compose command
  • mba logs postgres. show logs and tail of postgres container
  • mba compose logs -- --tail=100 -f postgres. same, using raw compose command
  • mba down. destroys the stack
  • mba ps. show all mba containers running in the system.
  • mba compose ps. show running containers from this stack
  • mba compose .... proxy to all docker-compose commands
  • mba compose config. show generated yml
  • mba nuke. kill all mba containers dead

mba wraps docker-compose, and using the command compose, it will pass all further commands to the underlying docker-compose.

mba compose config is equivalent to docker-compose -f tmp.yml config and will show the yml file that is generating with your current flags.

By default, it will share ports. ==–publish false= will jail the whole stack not publishing any port.

If current directory is not a metabase source dir, by default it will use dockerhub’s metabase/metabase image. This WILL start automatically.

If it is ran in a metabase source directory, it will run the .devcontainer image, so you should shell into it and run the commands manually to start the app.

SUPERHACK: ENV vars are passed around at the moment, to have the commands available

mba up
mba shell
lein repl     # or `eval $MBA_CLI`. lein repl as cider-jack-in would call it with
              # fixed host 0.0.0.0 and port 7888 (which is the exported port by mba)

mba shell
yarn build-hot # or `eval $MBA_YARN_BUILD`, as a tunnable command to
               # run the proper yarn build. It used to be a place to
               # put special flags like --host 0.0.0.0, but now the
               # flag is merged in master

so hacky, amazing.

  • mba –help may help you with the commands and option, but for the values, explore src/main.clj
  • to pass flags to docker-compose , or any command underlying mba, the way to pass as a parameter to inside is with --: mba --app-db postgres compose logs -- -f

Flags

**tl;dr**:

mba --mb ~/prj/metabase \
    --app-db postgres:13 \
    --data-db mariadb \
    --proxy nginx\
    --env FOO=bar\
    --network n1\
    --prefix mb1\
    up

–mb / -M

Main metabase app source or image.

If mba runs in a metabase source directory, it will default to the current directory. Else, it defaults to dockerhub:metabase/metabase-enterprise docker image.

Set it explicitly like the following:

  • mba --help will show the options accepted by mba.
  • cd ~/src/metabase; mba -M ./
  • mba --mb ~/src/metabase/
  • mba --mb file:~/src/metabase/
  • mba --mb docker:metabase/metabase:v0.37
  • mba --mb git:ref <- not implemented

–app-db / -d

App db for metabase. Defaults to postgres. Valid options are:

  • postgresql
  • postgres
  • postgres:13 Or wtv version
  • mysql
  • mariadb
  • h2

–data-db / -D

Datawarehouse db. Defaults to nil. Valid options are:

  • postgresql
  • mariadb
  • mysql
  • mongodb
  • vertica
  • presto
  • sparksql
  • sqlserver

–proxy

Proxy infrastructure. Defaults to nil. Valid options are:

  • nginx
  • haproxy
  • envoy

–env / -e

Injects env variables to the metabase container. Can be present multiple times. For example, this would start a metabase image with a custom h2 file:

mba --mb dockerhub:metabase/metabase\
    --env MB_DB_FILE=/root/testdb \
    --app-db h2\
    up

Remember that /root/testdb means ~/.mba/.mba-home/ from your host.

–publish

Publish ports. Boolean. Defaults to true.

–prefix / -p

stack name. Defaults to “mba-[somehash]”. Useful when running multiple instances.

–network / -n

Network name. By default it creates an anonymous network per stack, Fixing it is useful to make many metabase join the same network.

ports

By default, the stack will share ports:

  • 3000: backend
  • 8080: frontend
  • 7888: for lein repl
  • 1025: maildev mail port
  • 1080: maildev http
  • 8081 if –proxy is set

networks

By default it will create a new unnamed network, but we can tell mba to create a named network with -n foo. This and docker-compose run allows us to run different mb versions in the same environment to test upgrades and the like.

# this will start the "main" one
mba --mb docker:metabase/metabase:v0.37.9 --app-db postgres -n n1 up
# attach a new container to the network. Can run migrations.
# or hit http://metabase:3000/api/session/properties
mba --mb file:~/workspace/metabase --app-db postgres -n n1 run metabase bash

Note: you can’t run bash on a docker image that we provide (see metabase/metabase#14528), but you can run normally, and then exec into it and run sh. That works ok.

Troubleshooting

I’m using docker:metabase/metabase and mba is using an old version

Use mba --mb docker:metabase/metabase:v0.37.9 up or use the pull command: mba --mb docker:metabase/metabase pull

modifications done (or to be done) to make it work as a dev env

clojure

Add :port and :host.

lein repl :headless :host 0.0.0.0 :port 7888

How to see actual Metabase queries sent to the database

./src/main.clj --app-db postgres --data-db postgres logs postgres | grep -A 5 "queryHash"

cypress

TODO

how to see what is the actual compose being built

./src/main.clj --data-db postgres --app-db postgres compose config | less

See also

mba's People

Contributors

kidd avatar paoliniluis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

isabella232

mba's Issues

naming of services

should we have the same name for service type?

app-db, data-db, proxy, vs metabse, postgres:12, nginx.

pros:
simpler and more predictable basic structure.

cons:
if you want to use the same db for app-db and data-db, you can't reuse it. maybe it's not a bad thing

fix h2

decide if h2 has to have a container on its own or not.

if it has it , it's more symetric with others.
We have to take into account that 'sharing' networks might imply duplicated h2 containers if we default to :h2.

cant start server caus it locks the db file

fail to spin metabase-head

Reza reported:
mba --mb docker:metabase/metabase-head up the metabase container fails to start up with this error:

Using these JAVA_OPTS: -XX:+IgnoreUnrecognizedVMOptions -Djava.awt.headless=true -Dfile.encoding=UTF-8Error: Unable to access jarfile ./target/uberjar/metabase.jar

It seems that we build oficial releases and nightly builds in a slightly different way, the working dir of the official one is /, and the one for head is /app.

mba usually overrides working_dir to /app/source, where it mounts the source in case we're using it for dev. As it is not the case when runnning docker:something, we can not override working_dir.

mba should detect whether a tty is bound and set other parameters accordingly

As of now, the changes in the docker-exec-no-tty branch are needed in order to make tests from GitHub actions pass. On the current master branch, it results in an error like the input device is not a TTY when attempting the mba ... run steps. We should either make tty a parameter to mba (so CI tests can disable it), or else intelligently detect whether a TTY is bound and set the properties accordingly.

mba run

run a command externally without shelling in:

mba run lein run should just work

split app-db and data-db maps

The fact that all dbs live in the same map makes it easy to pick whatever we want from that bag and we do the validation in cli-options.

Maybe though, we should think of having different maps, to have a clearer split. Or at least, "mount" them as "app-db" and "data-db" services.

The corner cases arise when we want the same app-db and data-db.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.