Code Monkey home page Code Monkey logo

ansible-debootstrap's Introduction

Run debootstrap through ansible

This role will allow you to bootstrap a system over SSH using ansible. Typical applications include remote installing via bootable thumb drive or PXE boot or installing virtual machines using the host system. The author personally uses this for all his Debian installation needs.

Features

  • Your own partition layout: You can choose your own partition layout
  • Encryption: Encrypt partitions and hard drives with a pre-shared key (it is strongly suggested you use ansible-vault here)
  • ZFS is supported, even/especially as a root device

Limitations

  • You can't have the same names for ZFS pools and crypto devices
  • requires a /boot and BIOS boot partition for encrypted / ZFS

Supported distributions

  • Debian
    • Stretch
    • Buster (untested)
    • Bullseye (untested)
  • Ubuntu
    • bionic (18.04 LTS)
    • disco (19.04)
    • focal (20.04)
    • hirsute (21.04)
    • impish (21.10) (will install newer debootstrap)

Minor modifications will likely make it possible to install newer and perhaps older versions as well, or even other Debian based distributions. There are some sanity checks executed, depending on lists in vars/main.yml, so you would need to add the distribution codename there. Pull requests welcome.

Host system caveats

This will make a few modifications to the system that is being used as install medium. This includes installing a few packages, otherwise everything should be cleaned up afterwards.

  • It will install the eatmydata and debootstrap package
  • It will install cryptsetup for encrypted targets
  • It will install ZFS tools when using ZFS

Normally it is assumed that a some sort of PXE or USB rescue system is used, some caveats apply otherwise with regards to device names. As an example, you can't use the same name for any device mapper devices (luks encryption) or ZFS pools.

Configuration

Global variables

release: The release codename (required, example: cosmic)
root_password: Hashed, Salted root password, you can use mkpasswd to create one (Example: $1$sfQaZkVR$Vo/0pjmJaljzakEQFCr7Q/, obviously don't use this one ;) )
use_serial: Serial device to use for console / grub
use_tmpfs: Bootstrap to tmpfs, it's quicker and may reduce wear on flash (default: yes)
use_efi: In case the system supports UEFI, "grub-efi" will be installed on te target system, otherwise "grub-pc" (default: yes). This requires a VFAT partition available at /boot/efi.
kernel_cmdline: Anything you need/want to pass to the kernel (default: provided by distro)
layout: Dictionary of partitions / devices (required, see below)
md: List of DM-RAID devices (see below)
lvm: List of LVM volumes (see below)
install_ppa: PPAs to install (Ubuntu Only, see below)
install_packages: List of packages to install
dbstrp_zfs_backport Use the ZFS backport repo from https://launchpad.net/~jonathonf/+archive/ubuntu/zfs
zfs_pool: ZFS pool (see ZFS section)
zfs_fs: ZFS filesystems (see ZFS section)
zfs_root: ZFS devices to use as root
wipe: Set to to string "ABSOLUTELY" if you wish to wipe the disk, this will remove any disklabels present as well as issue a TRIM/UNMAP for the device. Useful when you want to overwrite a target. Please use extreme caution

Debootstrap user options

dbstrp_user:
name: A name of debootstrap user (default: debootstrap)
uid: UID of debootstrap user (default: 65533)
group: A group name of debootstrap user (default: name of debootstrap user)
gid: GID of debootstrap user (default: uid of debootstrap user)
password: A hashed password of debootstrap user (default: *)
non_unique: Ability to create non unique user (default: yes)

Partition Layout layout

Layout is a list of dictionaries, every dictionary representing a target device. The dictionary contains the device names, as well as another list of dictionaries representing partitions (as you can see, I like my complex data structures).

Elements in the device dictionary:
device: Path to the device
partitions: List of partition dictionary
skip_grub: boolean set to yes if you want to skip installing grub on this device (default False)

Partition dictionary

num: Ascending number, a limitation here, just make sure it increments with every element
size: size with qualifier (M for Megabytes etc.), leave empty for the rest of available space
type: GTID typecode, for example 8200 for linux, see table below for common type codes
fs: Target filesystem (optional, for example ext4, xfs, not needed for zfs)
mount: Where to mount this device (optional, example /boot)
encrypt: Set to yes if you want encryption
target: Target name for device mapper (required when using encryption, for example cryptroot)
label filesystem label to use
partlabel partition label to use, defaults to label if defined, otherwise no label will be set.

Type code Description
8200 Linux Swap
8300 Linux Filesystem
ef02 BIOS Boot partition (for grub)
fd00 Linux RAID
8e00 Linux LVM

DM-RAID devices

Within the md list you can set up DM RAID devices. List items are dictionaries supporting the following keys:

level: RAID level to be used (one of 0, 1, 4, 5, 6, 10) required
bitmap: Device to use for bitmap, default "internal", use "none" to disable bitmap
chunk_size: RAID chunk size (required for all RAID levels except 1)
name: Device name required
members: List of devices to add to RAID

Encryption and mount options are supported here. Please note that the order is

  1. RAID, 2. Encryption, 3. LVM, so it is currently not possible to create a RAID of two LUKS devices or encrypt an LVM volume.

Encryption Options

passphrase: Passphrase for encryption, use ansible-vault here please. (required when using encryption)
cipher: Encryption cipher (default aes-xts-plain64)
hash: Hash type for LUKS (default sha512)
iter-time: Time to spend on passphrase processing (default 5000) key-size: Encryption key size (default 256, values depend on cipher, for AES 128, 256, 512)
luks-type: LUKS metadata type (Default luks2)
luks-sector-size: Sector size for LUKS encryption (default 512, possible values: 512, 4096)
target: Device name to be used (required)

LVM configuration

LVM pvs can be created on disks, partitions as well as encrypted devices (use /dev/mapper/target for LUKS). This is a list of dictionaries, dictionary keys from partition dictionary can be used (except encryption) like mount, fs etc.

lvm: Dictionary of lvol options, these are passed to the lvol noduule as such, all options available to that module can be used.

Example device with partitions:

layout:
  - device: '/dev/sdb'
    partitions:
      - num: 1
        size: 1M
        type: ef02
      - num: 2
        size: 253M
        type: 8300
        fs: ext4
        mount: /boot
      - num: 3 # Notice absence of size here, will use full disk
        type: 8300
        fs: ext4
        mount: /

PPA to install, for Ubuntu install_ppa

Simple list of PPA to use, example:

install_ppa:
  - ppa:nils-nm/zfs-linux-unofficial

ZFS configuration

Pool definition zfs_pool

This defines the devices to use for the ZFS pool, you can use devices and targets defined in layout. A list of dictionaries with the following elements:

poolname: name of the ZFS pool (required, example rpool)
devices: List of devices, you can insert key words like mirror, raidz etc. like you would when using zpool create. (required, example below)
options: List of options for the pool
fs_options: Options for all filesystems in that pool

ZFS Filesystems / Datasets to create zfs_fs

This looks a lot like the pool definition above, again a list of dictionaries (they call me the one trick pony). Dictionary definition:

path: Path of the dataset, make sure that they are in the correct order, (required, example: rpool/root)
options: List of filesystem options, see examples

Set the root / boot fs for ZFS zfs_root

This is used when you want to use ZFS as your root filesystem. Set it to the dataset which you want to use as root, example: rpool/ROOT/Ubuntu

ZFS example:

This example is plucked from my own configuration, it will create a lot of datasets with different options and should give you a good overview over the possibilities.

zfs_pool:
  - poolname: rpool
    devices:
    - /dev/mapper/cryptroot
    options:
    - ashift=12
    fs_options:
    - canmount=off
    - mountpoint=/
    - compression=lz4
    - atime=off
    - normalization=formD

zfs_fs:
  - path: 'rpool/ROOT'
    options:
    - canmount=off
    - mountpoint=none
  - path: 'rpool/ROOT/ubuntu'
    options:
    - mountpoint=/
  - path: 'rpool/home'
    options:
    - setuid=off
  - path: 'rpool/home/root'
    options:
    - mountpoint=/root
  - path: 'rpool/var'
    options:
    - exec=off
    - setuid=off
    - mountpoint=legacy
    mount: /var
  - path: 'rpool/var/cache'
    options:
    - 'com.sun:auto-snapshot=false'
    - mountpoint=legacy
    mount: /var/cache
  - path: 'rpool/var/log'
    optons:
    - mountpoint=legacy
    mount: /var/log
  - path: 'rpool/var/spool'
    optons:
    - mountpoint=legacy
    mount: /var/spool
  - path: 'rpool/var/tmp'
    optons:
    - mountpoint=legacy
    mount: /var/tmp
  - path: 'rpool/var/lib'
    optons:
    - mountpoint=legacy
    mount: /var/lib
  - path: 'rpool/var/lib/dpkg'
    options:
    - exec=on
    - mountpoint=legacy
    mount: /var/lib/dpkg
  - path: 'rpool/srv'

zfs_root: 'rpool/ROOT/ubuntu'

Test playbook for vagrant

The directory meta/tests contains a test playbook, inventories and Vagrantfile for local testing. The vagrant box by default contains three devices, one for the source vagrant box and target devices for your install (/dev/sdb, /dev/sdc). To test your new installation you would have to switch boot devices in the SeaBIOS boot menu (easily achieved via the VirtualBox GUI). Currently, only VirtualBox is supported.

ansible-debootstrap's People

Contributors

doubleo8 avatar mvk15 avatar nilsmeyer avatar progval avatar rpw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

ansible-debootstrap's Issues

Add automated tests and test playbook

There should be a way to automatically test this, since it's a bit of a special use case may require some sort of special vagrant box (booted from secondary disk or so).

Include the simplest possible example in README

Could you please include a simple example on how to make a simple directory debootstrap with this script? That would make it more complete and easier to try out.

I may do a PR when (or if) I figure it out. Is it a supported scenario to do an installation to a directory? It is especially useful for LXC containers.

ZFS module issues with version 0.8

There is a problem with using ZFS from the jonathonf ppa - at some point the shipped and hopelessly outdated zfs module is loaded, preventing the newly installed/built module from loading, later on breaking ZFS support. The error looks like this:

failed: [ansible-debootstrap] (item={'poolname': 'rpool', 'devices': ['/dev/sdb3'], 'options': ['ashift=12'], 'fs_options': ['canmount=off', 'mountpoint=/', 'compression=lz4', 'atime=off', 'normalization=formD']}) => {"ansible_loop_var": "item", "changed": true, "cmd": ["zpool", "create", "-o", "ashift=12", "-O", "canmount=off", "-O", "mountpoint=/", "-O", "compression=lz4", "-O", "atime=off", "-O", "normalization=formD", "-R", "/mnt/bootstrap", "rpool", "/dev/sdb3"], "delta": "0:00:00.011030", "end": "2019-06-25 23:19:50.791048", "item": {"devices": ["/dev/sdb3"], "fs_options": ["canmount=off", "mountpoint=/", "compression=lz4", "atime=off", "normalization=formD"], "options": ["ashift=12"], "poolname": "rpool"}, "msg": "non-zero return code", "rc": 1, "start": "2019-06-25 23:19:50.780018", "stderr": "cannot create 'rpool': invalid argument for this pool operation", "stderr_lines": ["cannot create 'rpool': invalid argument for this pool operation"], "stdout": "", "stdout_lines": []}

The problem is a version mismatch between the module and the installed zfs utils. Using the old ZFS version works.

Support zfs legacy mountpoints

Since there are some problems with automounting ZFS on /var, it should be possible to use legacy mountpoints and add those filesystems to fstab

[Question] LVM for VMs

I have a volume group (vg0) on a vm host platform, which has all vm root partitions as volumes (/dev/vg0/testvm-root). The vm sees the volume as /dev/vda. I've been trying for a while now, but can't figure out a way to handle this with this role. Is it possible without modifying this too much or do I basically need to reimplement the handling of partitions?

Creating the swap area in a logical volume (LVM)

Hi,
I really love your role Ansible! It is particularly suitable for my setup where the Debian installer can not do it.

Could you add swap support under LVM (mkswap and append to fstab) please ?

Here are my dictionaries:

release: buster
layout:
  - device: '/dev/sda'
    partitions:
      - num: 1
        size: 1M
        type: ef02
        label: BIOS
      - num: 2
        size: 100M
        type: ef00
        fs: vfat
        mount: /boot/efi
        mountdump: 0
        mountpass: 1
        mountopts: noatime,nofail,x-systemd.device-timeout=1,umask=0077
        label: EFI
      - num: 3
        size: 250M
        type: 8300
        fs: xfs
        mount: /boot
        mountdump: 0
        mountpass: 2
        mountopts: noatime,nodev,nosuid
        label: BOOT

lvm:
  - lv: swap
    vg: vg-srv
    pvs: /dev/sdb
    size: 1G
    opts: ~
    label: swap
    fs: swap
    mount: none
    mountdump: 0
    mountpass: 0
    mountopts: sw
  - lv: os
    vg: vg-srv
    pvs: /dev/sdb
    size: 15G
    opts: ~
    label: os
    fs: xfs
    mount: /
    mountdump: 0
    mountpass: 1
    mountopts: noatime,nodev

Thank you in advance

"cryptsetup" tool uses early then it is installed

The first Ansible task in the file encryption.yml uses the command cryptsetup --version before it will be installed in the 3rd task:

---

- name: fetch cryptsetup version  <==========================
  command: "cryptsetup --version"
  changed_when: no
  register: _cryptsetup_version

- name: register version
  set_fact:
    cryptsetup_version: "{{ _cryptsetup_version.stdout[11:] }}"

- block:
  - name: install dependencies  <============================
    apt:
      name: "{{ dependencies.encryption }}"
      state: latest
      update_cache: no # skip cache check since we already ran apt

playbook fails creating /etc/fstab

trying to set up a system with 2 disks in mdadm raid, playbook gets to place templates in /etc and fails:

failed: [ip_address] (item={'src': 'fstab'}) => {"ansible_loop_var": "item", "changed": false, "item": {"src": "fstab"}, "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'fs'"}
changed: [ip_address => (item={'src': 'crypttab', 'dest': 'crypttab'})

I believe because /dev/sd{a,b}1 has no filesystem (it's a boot partition)

~# lsblk -f
NAME      FSTYPE      LABEL                UUID                                 MOUNTPOINT
loop0     squashfs                                                              /lib/live/mount/rootfs/img.current.sq
sda                                                                             
├─sda1                                                                          
└─sda2    linux_raid_ 163-172-251-100:root 5d8518c2-7589-668b-d736-6c8a724d21a5 
  └─md127 ext4                             9a54cc12-4af5-43af-859b-59e23735ffbd /mnt/bootstrap
sdb                                                                             
├─sdb1                                                                          
└─sdb2    linux_raid_ 163-172-251-100:root 5d8518c2-7589-668b-d736-6c8a724d21a5 
  └─md127 ext4                             9a54cc12-4af5-43af-859b-59e23735ffbd /mnt/bootstrap

Round down LUKS partition to luks-sector-size boundary

device mapper has issues when the partition size doesn't divide cleanly by luks-sector-size. This is not an issue with devices that report a native sector size of 4KiB, but may be an issue otherwise. The partition size can be rounded down to eh 4KiB boundary so this problem does not occur - there probably needs to be some wiggle room for other data structures as well.

The error one sees with this is: device-mapper: reload ioctl on failed: Invalid argument
And in the kernel log:

device-mapper: table: 253:2: crypt: Device size is not multiple of sector_size feature
device-mapper: ioctl: error adding target to table

Add support for dropbear initramfs

For remote system it makes sense to use dropbear in initramfs, , since this may be required for first boot when there is no iKVM available.

  • install dropbear package
  • convert newly generated ssh keys to dropbear format
  • rebuild initramfs

Stretch goal:

  • Add some sort of remote unlock script

Convert to two stage playbook to enable running user roles

Instead of running a lot of things via the shell module, invoking chroot, I think it may be more prudent to at least temporarily start an sshd instance in the chrooted system (on a different port or possibly tunneled through the main host), then allow further plays to be executed on that sshd instance. This would also allow the user to add their roles to the playbook and allows separating out some of the more advanced features.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.