Code Monkey home page Code Monkey logo

slipway's Introduction

Slipway

An easier way to use containers for development on Linux. It automatically maps credentials, sets up things such as the clipboard, and corrects permission issues which arise from developing with containers natively.

What this tool does

  • Automatically maps ssh credentials into the container
  • Automatically maps GPG
  • Inspects the image for volumes and will correct any permission issues. For example if you're using npm and you want the cache to be persisted between restarts you can add the following to your image:
VOLUME $HOME/.npm

Since docker will create the directory with root as the owner slipway will correct it automatically.

  • Detects X11 support and will map it into the container (clipboard integration).
  • Handles open calls (e.g., open in browser on the host) via unix sockets.
  • Maps your ~/workspace directory into the container (can be overriden).
  • Sets the timezone to match the host
  • Sets your git config to match the host
  • Maps credentials files for certain package managers to the container (yarn, cargo, etc).
  • sets up correct uid mappings on podman.

Requirements

  • Linux OS
  • Python 3.8+
  • Podman (rootless only)

Getting Started

Install slipway:

python3 -m pip install slipway

Run an example image:

slipway start aghost7/nodejs-dev:focal-carbon

Configuration

The start command line options can be specified in a configuration file under ~/.config/slipway.yaml.

pull: true
pull_daily: true
runtime: podman
alias:
  devops:
    image: aghost7/devops:focal
    network: slirp4netns
    environment:
    - AWS_ACCESS_KEY_ID

You can then use your devops alias in place of the image name:

slipway start devops

Optional GnuPG (GPG) Support

On your host, you will need to have gpg configured with the daemon running. Slipway will detect that gpg is running and will automatically create a bind mount (volume) to map the socket file into the container.

Enable gpg signing git commits:

git config --global commit.gpgSign true

If you want to always sign tags:

git config --global tag.forceSignAnnotated true

Since we want gnupg to be used from the terminal interface, we need to change the configuration under ~/.gnupg/gpg.conf:

use-agent
pinentry-mode loopback

Using rootless containers

Slipway supports podman, which is an alternative implementation to docker that has much better security. There are additional steps to setting this up, which is why it isn't the default.

Start by installing podman.

Setup the registry configuration:

mkdir -p ~/.config/containers
echo 'unqualified-search-registries = ["docker.io"]' > ~/.config/containers/registries.conf

Install some additional dependencies:

sudo apt-get install -y fuse-overlayfs slirp4netns

Grant your user some [subuids][subuids]/[subgids][subgids]:

echo "$USER:100000:600000" | sudo tee -a /etc/subuid
echo "$USER:100000:600000" | sudo tee -a /etc/subgid
podman system migrate

And then you can run your containers with podman instead!

slipway start --runtime podman aghost7/nvim:focal

I can't use networking tools (nmap, traceroute, etc) with rootless containers

This is actually because slipway defaults to host-based networking. When using rootless containers, you need to change the network used to slirp4netns.

slipway start --network slirp4netns aghost7/devops:focal

Developing

Requirements:

  • python 3
  • poetry

Install dependencies:

poetry install

Run tests:

poetry run pytest

slipway's People

Contributors

aghost-7 avatar

Stargazers

Matthew Pengelly avatar Mark avatar  avatar  avatar Harry Jubb avatar Marek Wywiał avatar  avatar Daniel Felt avatar Christian Werner avatar  avatar Marc Brooks avatar David MacEachern avatar Shinkiro avatar  avatar Julian Mazzitelli avatar Emmanuel avatar

Watchers

James Cloos avatar  avatar

slipway's Issues

Bind mount permission errors

If the --volume is bind mount, it will create the directory automatically if it does not exist. Issue with this is it will set the ownership to root instead of the current user (which is what I would expect normally). Solution would be to maintain a list of all bind mounts and create directories which are missing. I have a mount for the gitconfig file, not sure how I'm going to handle this. It looks like there's an XDG compliant way to set the configuration for git, will need to test it out.

strace

Since the capability is disabled in docker by default, I may need to allow it when running using slipway.

Add support for bindfs

Right now everything works fine since the UID for the user in the container and host are the same. However, I already ran into issues with travis where the UID on the host and container don't match. It seems like the simplest way would be to use a volume plugin for bindfs. Since it requires the plugin to be installed I'll need to put it behind a flag and have the feature turned off by default.

https://github.com/lebokus/docker-volume-bindfs

Download progress bar

When it attempts to pull down an image, it doesn't show the progress of the download to the user. This is poor UX.

Automatically detect correct uid / gid mappings (uidmap)

Steps to reproduce

  • user inside of container has uid of 1001
  • user on host has a uid of 1000
  • run slipway start <the image>
  • run touch ~/workspace/foobar

Expected behaviour

The file is created.

Actual behaviour

Permission denied.

Notes

The --userns=keep-ids should be replaced with a more advanced, custom logic.

--pull option

This will make slipway attempt to update the image before creating the container.

Migrate to jenkins

I dont think I should be staying on travis ci for very long...

  • run test suites against docker and podman
  • run mypy
  • run black

GPL

Switch of license.

Handle volume mapping when using xdg-open

Steps to reproduce:

  • have html file in workspace inside of container
  • run xdg-open ./*.html from container

expected behaviour:
I should see the file opened in my browser.

mac support

  • support tcp socket for command proxy
  • allow command proxy to support additinal command (e.g., pbcopy / pbpaste)
  • change volume automapping to rely on the volume information fom the host instead of mountinfo inside of the container
  • remove docker blacklist
  • disable gpg support on mac
  • disable uidmapping on docker
  • disable x11 mapping on mac
  • update tests to run on mac
  • add mac build to github actions
  • ???

OSX?

Since developing through docker for mac isn't an option (file system share is too slow to be usable), maybe I could wrap around vagrant + vagrant-nfs_guest.

Configuration file

slipway start --pull aghost7/nodejs-dev:bionic-carbon isn't exactly a short command to run. I want the ability to:

  • put any option into a configuration file
  • add aliases, e.g., map slipway start node to slipway start aghost7/nodejs-dev:bionic-carbon

gnupg support

Would be nice if I could sign using pgp my commits.

edit: Looks like you can set it up using the socket file instead. Tried a different way in the past, which failed, could be the way to go.

xdg-open integration

:Gbrowse doesn't work at the moment. This is due to xdg-open not being able to run within the container, since the browser is on the host. Will need to write a bridge similar to snipsnip.

Integrate GNOME keyring

Being able to call GNOME keyring to store password from the container would make storing keys in environment variables less of an issue.

Share subcommand?

Similar to vagrant's share subcommand.

Would be very handy to have something which can initialize an ssh session. There is wemux which can make it easier to administrate multiple sessions more easily but I need something to generate the ssh configuration automatically.

Starting requirements:

  • Each client session gets its own key
  • The access is revoked once the container being shared is removed
  • Server can unshare
  • Client should be able to connect to the session using a single command. It is possible for that single command to be a call to slipway. Preferably it should be a call to the ssh command, since everyone has ssh installed on their machine.

Phase 2

I also need to traverse multiple networks in some cases, so probably will require some sort of tunneling / bridging in place. When you use vagrant share, you're instead given a url to a remote server which acts as a bridge between both networks.

Extra

I think creating a guest user automatically without the sudo groups would be great, but for now I think I'll just share the session as the same user. This is a feature so that I can share ssh sessions with my friends so there isn't a security issue here.

Drop support for rootful docker

Rootless docker seems to work well enough. Since I don't want to promote bad security practices, I will remove support for it.

New volume permission errors

When it initially creates the container, I'm having issues where the volume ownership is incorrect (set as root).

What I could do is when I run the volume creation step, I create a container just to set the permissions based on the owner of the parent directory.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.