setup-da-distro

A framework to manage programs from web sources for non-privileged users on Linux systems

Motivation

During occasional strolls on reddit or github, my attention is often drawn towards programs that increase productivity or provide an enhancement over others. (As a somewhat irrelevant side note - these programs mostly work in the command line.) Usually these programs are available for download as binary or script, meaning that naturally, the management (installation, upgrade, removal) of those programs has to be performed manually. At this point sdd comes into play: It provides a framework to automatize the tasks of managing the programs (or, in sdd terminology, 'apps'). The procedures to manage specific apps are defined within scripts in this repository (at lib/sdd/apps/user/).

sdd enables me to keep track of my favorite programs, on different machines. I'm working towards having systems set up in a reproducible way on my machines. sdd helps me, since I might have different Linux distributions installed on these machine, with different package manager providing different versions of required programs (or none at all). I can freeze the versions of all apps managed by sdd with sdd list --installed > sdd_freeze.txt, and re-create them with sdd install <(cat sdd_freeze.txt | xargs).

Finally on some systems I might not have root access, and hence I can't install missing programs using a package manager. sdd allows for unprivileged management of programs.

WARNINGS

sdd is a simple collection of bash scripts, not a mature package manager (neither do I aim to turn it into one...). Using it might break things on your system (e.g. overwrite existing program files).

When using sdd, you execute functionality to manipulate your system. Especially, you download programs from third parties, and install them on your system. Most sources are provided by GitHub releases pages. Keep in mind that repositories can be compromised, and malicious code placed inside; and sdd will still happily download it. (If you have an idea how to mitigate this security flaw, please open an issue.)

sdd is targeted to 64bit Linux systems. Some apps might not work when installed to different architectures. If available, bash and/or zsh shell completion and man pages are set up when installing an app.

Demo

The following screencast demonstrates how sdd is used to install, upgrade and uninstalled the fd utility.

Installation

Clone the directory and run the bootstrap script to install sdd to ~/.local:

git clone https://github.com/pylipp/sdd
cd sdd
git checkout v0.1.1.0  # or any other revision
./bootstrap.sh

You can specify the installation directory with the PREFIX environment variable:

PREFIX=/usr ./bootstrap.sh

Please verify that the bin sub-directory of the installation directory is present in your PATH. You might want to append this to your shell configuration file:

export PATH="~/.local/bin:$PATH"

Same applies for the MANPATH:

export MANPATH="~/.local/share/man:$MANPATH"

For enabling zsh completion functions (oh-my-zsh users: put this before the line that sources oh-my-zsh.sh since it calls compinit for setting up completions):

fpath=(~/.local/share/zsh/site-functions $fpath)

For enabling bash completion functions, you should be fine if you already use the bash-completion package. Otherwise add this snippet to your ~/.bashrc:

if [ -d ~/.local/share/bash_completion ]; then
    # Source custom completion files
    while IFS= read -r -d '' f; do
        . "$f"
    done < <(find ~/.local/share/bash_completion -type f -print0)
fi

sdd is tested with bash 4.4.12.

Dependencies

sdd depends on bash, git, and wget. Python-related apps require python3.

Upgrading

Once the program is bootstrapped, upgrade to the latest version by

sdd upgrade sdd

Usage

Installing an app

Install an app to SDD_INSTALL_PREFIX (defaults to ~/.local) with

sdd install <app>

You can specify a custom installation prefix like this:

SDD_INSTALL_PREFIX=~/bin sdd install <app>

or by exporting the SDD_INSTALL_PREFIX environment variable.

By default, sdd installs the latest version of the app available. You can specify a version for installation:

sdd install <app>=<version>

This command overwrites an existing installation of the app without additional conformation.

The format of the <version> specifier depends on the app that is managed (usually it's the tag of the release on GitHub).

Upgrading an app

To upgrade an app to the latest version available, run

sdd upgrade <app>

If you want to upgrade to a specific version, run

sdd upgrade <app>=<version>

Internally, sdd executes un- and re-installation of the app for upgrading unless a specific upgrade routine has been defined. The usage of SDD_INSTALL_PREFIX is the same as for the install command.

Uninstalling an app

To uninstall an app, run

sdd uninstall <app>

The usage of SDD_INSTALL_PREFIX is the same as for the install command.

Batch commands

The commands install, upgrade, and uninstall can take multiple arguments to manage apps, e.g.

sdd install <app1> <app2>=<version> <app3>

Listing app management information

List installed apps by running

sdd list
sdd list --installed

List all apps available for management in sdd with

sdd list --available

List all installed apps that can be upgraded to a more recent version with

sdd list --upgradable

The list command options come in short forms, too: -i, -a, -u

General help

High-level program output during management is forwarded to the terminal. Output of the sdd_* functions of the app management file is in /tmp/sdd-<command>-<app>.stderr. For increased verbosity when running sdd, set the respective environment variable before invoking the program

SDD_VERBOSE=1 sdd install <app>

You can always consult

sdd --help

Apps available

In alphabetical order:

Customization

You can both

• define app management files for apps that are not shipped with sdd, and
• extend app management files for apps that are shipped with sdd.

The procedure in either case is:

1. Create an empty bash file named after the app in ~/.config/sdd/apps (without .bash extension).
2. Add the functions sdd_install and sdd_uninstall with respective functionality. It's mandatory to add a function, even if without functionality (define sdd_uninstall() { return; }).
3. Optionally, you can add an sdd_upgrade function. It will be executed for upgrading, instead of sdd_uninstall followed by sdd_install.
4. You're able to manage the app as described in the 'Usage' section. sdd tells you when it found a customization for the app specified on the command line.

For exemplary files, see my personal definitions and extensions here.

Project structure

It is distinguished between

• framework files,
• app management files,
• testing files, and
• project meta-files.

Description

1. Framework files contain the logic to run the program. They provide generic utility methods (generating symlinks, reading environment variables, etc.). Examples are: program executable, library files.
2. App management files contain instructions to manage specific apps. For each app, at least one management file exists. A management file contains at least methods for installing, upgrading, and uninstalling an app. Management files are organized in directories indicating their level (user or root).
3. Testing files cover the functionality of both the framework and the app management files. They are executed in an isolated environment.
4. Project meta-files comprise documentation files, configuration files for development tools, an installation script, among others.

Contributing

Requirements

• git
• docker
• python3
• optionally: docker-compose

Development environment

Clone this repository and pull the Docker image to enable testing and style-checking.

git clone https://github.com/pylipp/sdd
docker pull pylipp/sdd

Set up the local environment for style-checking using the pre-commit framework.

test/setup/venv

Testing

The program is tested in a container environment using the bats framework. Invoke the test runner by

test/run.sh

You might want to skip app tests since they require an internet connection

NO_APP_TESTS=1 test/run.sh

For attaching to the test container after the tests have completed, do

test/run.sh --debug

For running specific tests (paths relative to repository root)

test/run.sh test/apps/fd.bats test/apps/sdd.bats

For creating a Docker container and attaching it to the terminal, do

test/run.sh --open

For style checking, run

test/run.sh --style

For building the image, run

docker build test/setup -t sdd:latest

You can also build and test the image in a single command by

docker-compose -f test/setup/docker-compose.test.yml up

Note that DockerHub automatically builds the image when source code is pushed to GitHub, and pushes it the DockerHub repository if the tests succeeded. The tests are defined in test/setup/docker-compose.test.yml.

Extending

You're looking for managing an app but it's not included in sdd yet? Here's how contribute an app management script:

1. Fork this repository.
2. In your fork, create a feature branch.
3. Create an empty bash file named after the app in lib/sdd/apps/user.
4. Add a test in test/apps/<app>.bats, e.g. verifying the version of the app to be installed.
5. Add the functions sdd_install and sdd_uninstall with respective functionality.
6. Add app name, link, and description to the table of available apps in the README file.
7. Add the new files, commit, and push.
8. Open a PR!

Releasing

1. Update Changelog.
2. Run ./release VERSION

Related projects

Note that maintaining packages (deb, rpm, etc.) might still require root privileges, depending on your system.