MIDI: Development environment for machine learning

Manage your machine learning environments, easily.

🐍 No docker, only python - Write python code to build the development environment, we help you take care of Docker.

📟 Built-in jupyter/vscode - Provision jupyter notebooks and vscode remote in the image, remote development is possible.

👨‍💻 Manage code and data - Help you manage the source code and dataset in the environment

⏱️ Save time - Better cache management to save your time, keep the focus on the model, instead of dependencies

☁️ Local & cloud - Run the environment locally or in the cloud, without any code change

🐳 Container native - Leverage container technologies but no need to learn how to use them, we optimize it for you

🤟 Infrastructure as code - Describe your project in a declarative way, 100% reproducible

Install

Linux

sudo /bin/sh -c 'wget https://github.com/tensorchord/midi/releases/latest/download/midi-linux-amd64 -O /usr/local/bin/midi && chmod +x /usr/local/bin/midi && /usr/local/bin/midi bootstrap'

Why use MIDI

How MIDI works

Documentation

Get started

A simple example

base(
    os = "ubuntu",
    lang = "python3.6"
)

shell("zsh")

apt_source("""
deb https://mirror.sjtu.edu.cn/ubuntu focal main restricted
deb https://mirror.sjtu.edu.cn/ubuntu focal-updates main restricted
deb https://mirror.sjtu.edu.cn/ubuntu focal universe
deb https://mirror.sjtu.edu.cn/ubuntu focal-updates universe
deb https://mirror.sjtu.edu.cn/ubuntu focal multiverse
deb https://mirror.sjtu.edu.cn/ubuntu focal-updates multiverse
deb https://mirror.sjtu.edu.cn/ubuntu focal-backports main restricted universe multiverse
deb http://archive.canonical.com/ubuntu focal partner
deb https://mirror.sjtu.edu.cn/ubuntu focal-security main restricted universe multiverse
""")

install_package(
    name = ['vscode', 'gdb>9', 'g++'],
    optional = True
)

if flags.cuda == True:
    install_package("cuda")

git_repo(
    local = "/code/repo",
    remote = "[email protected]:gaocegege/test.git",
)

ssh()

before_stop(
    git_push = True,
)

run_entrypoint(
    cmd = "python main.py",
    cwd = "/code/repo",
)

$ MIDI up
Installing dependencies...
Setting up IDEs...
Syncing data...
Finished! 😀
$ project-1 > pip install torch // Already in the docker 
Install Torch ....
$ project-1 > exit
$ MIDI down
You've installed torch in the dev environment, do you want to add it to the env file? [y/n] 
# Create customized zsh config
$ MIDI up --fresh # start a fresh dev environment
$ fresh > sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
$ fresh > vi ~/.zshrc 
# Change some personal config
$ fresh > exit
$ MIDI generate package --fresh
# We get all files modified in the docker by docker diff and pack it into a single file, which can be used later in other container

More examples

Features

from envd.

I think we can close it now, although our README is still too simple, sometimes naive.

from envd.