cybershadow / aconfmgr Goto Github PK

View Code? Open in Web Editor NEW

1.0K 19.0 39.0 1.04 MB

A configuration manager for Arch Linux

Shell 94.95% Roff 1.76% Makefile 2.59% Dockerfile 0.36% Ruby 0.34%

arch-linux configuration-management

aconfmgr's People

Contributors

Stargazers

Watchers

aconfmgr's Issues

[Feature request] Add option to skip md5sum check when saving configuration

After getting used to aconfmgr, I investigated why it took so long to save the configuration of my system, as I'm tend to save it quite often (when I make changes to the system I prefer to save the configuration and commit the changes ASAP so I have it fresh in my memory).

Most of the time was spent in the "Searching for modified files..." phase, and after some inspection of aconfmgr's source code, I found that it runs paccheck with the --md5sum option, which means that every time the configuration is saved, aconfmgr hashes the contents of all files of all the packages installed in the system, i.e. through multiple gigabytes of files.

If I remove the --md5sum flag from the paccheck call, aconfmgr is still able to detect changes in the package files, as paccheck still detects them due to the modification time and/or the size of the files when altering the package files in a reasonable way. This way, the execution time for saving the configuration halves in my system, here is an informal benchmark of it:

Cold run (drop caches), no md5sums: 36s
Cold run (drop caches), md5sums: 1m 45s
Warm run, no md5sums: 29s
Warm run, md5sums: 50s

Root partition is 18GB (most of it packages), in an SSD, and enough RAM to hold it entirely

I suppose running with the --md5sum option is intended (and necessary) to make 100% sure that the package has not been modified since it was installed. However, I think running a hash check every time one wants to save the configuration is overkill, and noticeably increments the execution time. I have been running my system for 3 months without the flag and after re-enabling it, no change was detected. Similarly, tools like rsync even default to checking for modification time and size and only do checksums if explicitly specified.

To start with, what do you think about adding a flag (e.g. --no-md5sum) that avoids checking the checksums of the packages when saving the configuration? I'll look into sending a PR if I can find some time, so I'm mostly interested in feedback.

Problem with sudo in combination with fingerprint reader

: Collecting data...
:: Compiling user configuration...
::: Using configuration in /home/mj/.config/aconfmgr
::: Sourcing /home/mj/.config/aconfmgr/10-ignore.sh...
::: Done (0 native packages, 0 foreign packages, 0 files).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating managed files...
:::: Done.
::: Searching for lost files...
:::: Done (426 lost files).
::: Searching for modified files...
........
:::: Done (39 modified files).
::: Reading file attributes...
:::: Reading file types...
stat: cannot stat 'Swipe your finger across the fingerprint reader': No such file or directory
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1 [AconfCompileSystem]
::::: /usr/lib/aconfmgr/common.bash:452 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/bin/aconfmgr:164 [Main]
::::: /usr/bin/aconfmgr:178 [main]

The message "Swipe your finger across the fingerprint reader" normally appears when i run sudo. So this error does not happen when i run sudo before the 'aconfmgr save' command.

Handling version mismatches

Currently, aconfmgr does not upgrade packages to the latest version when running aconfmgr apply. This means that running aconfmgr apply on two different systems can result in inconsistent states, since the systems might have different versions of the same package. It would be nice if aconfmgr automatically kept packages up-to-date.

One way would be to run pacman --sync --needed for every package in the configuration, rather than trying to determine manually which packages are already installed. Of course, it would also be necessary to run pacman --sync --refresh first.

I think this would also be a significant performance win. Instead of calling pacman once for every package, one could just use something like:

PrintArray packages | xargs -s 1024 pacman --sync --needed

Thanks for taking a look at this! I may have time to implement this myself, if there's interest.

stack trace error during `apply`

==> Installing missing dependencies...
error: target not found: pacli
==> ERROR: 'pacman' failed to install missing dependencies.
:: fzf is a new orphan package
:: failed to build bmenu package(s)
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1 [AconfInstallForeign]
::::: /usr/lib/aconfmgr/apply.bash:238 [AconfApply]
::::: /usr/bin/aconfmgr:167 [Main]
::::: /usr/bin/aconfmgr:178 [main]

After a save state, I uninstalled several native packages, and two foreign ones pacli and bmenu. re-installation of native packages was uneventful. Above error happened during foreign package re-installation.

contents of 00-config.sh:

verbose=1
prompt_mode=paranoid
aur_helper=pacaur
color=always

[Question] Dealing with different file contents in multiple systems

Hi! First of all, excellent software! Thank you for your work!

I hope you can provide some insight on a question I have.
In the README, you provide a solution for having different packages for different systems:

AddPackage coreutils
# ... more common packages ...

if [[ "$HOST" == "home.example.com" ]]
then
	AddPackage nvidia
	AddPackage nvidia-utils
	# ... more packages only for the home system ...
fi

I wonder how you'd deal with having different content for the same file in different systems.
As an example, consider that you have two systems (s1 and s2), each with a different /etc/hosts. Further suppose that you don't wish to have a different Git branch for each system.

My idea on how to solve this would be to have three folders where I store the files: files (the folder used by aconfmgr), files-s1, and files-s2. Then do something such as:

ln -sf "files-$HOST/etc/hosts" files/etc/hosts
CopyFile /etc/hosts

What do you think of this approach? I haven't tested it, would it work? How would you make it better? Any alternative ideas?

Another alternative would be for you to change CopyFile to allow distinct source/target paths; allowing me to have in the files directory a etc/hosts-s1 and etc/hosts-s2 and then do: CopyFile "/etc/hosts-$HOST" /etc/hosts.

In any case, I think the answer to this question should be added to the README; I don't think it is unreasonable for different systems to have different files.

Once again, thank you for your work!

Attempting to remove a directory as if it were a file

Found a minor bug:

aconfmgr save
: Collecting data...
:: Compiling user configuration...
::: Using configuration in /root/.config/aconfmgr
::: Sourcing /root/.config/aconfmgr/10-ignores.sh...
::: Sourcing /root/.config/aconfmgr/99-unsorted.sh...
rm: cannot remove '/tmp/aconfmgr-root/output/files//usr/lib': Is a directory
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/helpers.bash:251 [RemoveFile]
::::: /root/.config/aconfmgr/99-unsorted.sh:71864 [source]
::::: /usr/lib/aconfmgr/common.bash:100 [AconfCompileOutput]
::::: /usr/lib/aconfmgr/common.bash:714 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/lib/aconfmgr/main.bash:160 [Main]
::::: /usr/lib/aconfmgr/main.bash:177 [source]
::::: /usr/bin/aconfmgr:26 [main]

It looks like it was trying to remove a directory I set to be ignored as if it were a file. Looks like it needs a check to see if the object to remove is a directory or a file before attempting to remove it.

unable to execute /usr/bin/find: Argument list too long

After running aconfmgr apply following error appears:

: Collecting data...
:: Compiling user configuration...
::: Using configuration in /home/lpp/.config/aconfmgr
::: Sourcing /home/lpp/.config/aconfmgr/10-ignores.sh...
::: Done (0 native packages, 0 foreign packages, 0 files).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating managed files...
:::: Done.
::: Searching for lost files...
sudo: unable to execute /usr/bin/find: Argument list too long
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1 [AconfCompileSystem]
::::: /usr/lib/aconfmgr/common.bash:452 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/bin/aconfmgr:164 [Main]
::::: /usr/bin/aconfmgr:178 [main]

minor edits to readme + random notes

in readme, i miss the closing quote in

echo '127.0.1.1 laptop.example.net laptop >> "$f"

bash heredoc format

in section Inlining files, please add more samples for the bash heredoc format, including ...

... quoted heredoc format, with 'No parameter expansion, command substitution, arithmetic expansion, or pathname expansion'

# append constant string to file
# variables are not expanded
# note the quotes around the first EOF
cat >>/etc/profile <<'EOF'
if [[ ! $DISPLAY && $XDG_VTNR -eq 1 && $UID -eq 1000 ]]
then
        exec startxfce4
fi
EOF

... unquoted heredoc format, with both constants and expressions

# template variables
cmd_start_x_session=startxfce4
my_uid=$UID
my_vtnr=1

# template
# no quotes around first EOF
cat >>/etc/profile <<EOF
if [[ ! \$DISPLAY && \$XDG_VTNR -eq $my_vtnr && \$UID -eq $my_uid ]]
then
        exec $cmd_start_x_session
fi
EOF

the result is identic, but in version 2, you can use 'template variables'

ignore.sh samples

please add one [or more] ignore.sh sample files, like here

i would add, for example ...

IgnorePath '/usr/share/*'
IgnorePath '/usr/share/webapps/*'

IgnorePath '/var/lib/*'
IgnorePath '/var/lib/swap*' # swapspace files
IgnorePath '/var/swap*' # static swap files

IgnorePath '/usr/lib/*'
IgnorePath '/opt/*' # might be too strict

IgnorePath '/usr/libexec/darling/*' # darling, the macOS emulator
IgnorePath '/var/log/*'
IgnorePath '*/lost+found/*'

IgnorePath '/usr/local/include/*'
IgnorePath '/usr/local/lib/*'
IgnorePath '/var/tmp/*'

find -regex, instead of find -wholename

let me append custom filters to ignore_args, like ...

IgnorePathArgs -regextype posix-extended -regex '.*[.-][a-zA-Z0-9_-]{11}\.(mp4|mkv)'
IgnorePathArgs -regextype posix-extended -regex '/home/[^/]+/\.(cache|local/share|local/lib|thumbnails|npm/_cacache|ipfs|gnupg|purple/logs)/.*' \)
IgnorePathArgs -size +1M # ignore files of size > 1MebiByte = 1048576 Byte

i am aware, that scanning /home is out-of-scope for aconfmgr.
these are just samples, for what PE-regex can look like.

filtering by file size could be more useful.

perl regex

problem: find from findutils does not support perl regex.
i need perl regex, to use 'negative lookahead' (?!pattern), to ...

exclude /opt/*
and re-include /opt/i2p/router.config and /opt/i2p/.i2p/router.config

... for example.

workaround: use perl to filter the output of find

{
cat <<EOF
/opt/exclude-me
/opt/i2p/.i2p/i2psnark/exclude-me
/opt/i2p/.i2p/include-me.config
/opt/i2p/include-me.config
/etc/include-me
EOF
} | perl -ne 'print if not m#^/opt/(?!i2p/[^/]+\.config$|i2p/\.i2p/[^/]+\.config$)#'

to un-ignore paths:

function UnIgnorePath() {
	#global ignore_paths
	local delete="$@"
	local result=()

	while read -d $'\0' ignore_path
	do
		if [[ ! "$ignore_path" = "$delete" ]]
		then
			result+=("$ignore_path")
		fi
	done < <(
		printf '%s\0' "${ignore_paths[@]}"
	)
	ignore_paths=("${result[@]}")
}

ignore_paths should be unique:

	while read -d $'\0' ignore_path
	do
		ignore_args+=(-wholename "$ignore_path" -o)
	done < <(
		printf '%s\0' "${ignore_paths[@]}" \
		| uniq -z -u
	)

... so to include some files in /home

UnIgnorePath '/home'

{
cat <<EOF
/etc/include-me
/home/u1/.bashrc
/home/u2/.cache/medit-1/include-me
/home/u3/.cache/exclude-me
/home/u4/src/exclude-me
EOF
} | perl -ne 'print if not m#^/home/[^/]+/(?!\.bashrc$|\.cache/medit-1/)#'

bash exec magic

[ this is more a note to my self ]

this is called: bash automatic file descriptor allocation

as i found out after some time of searching

in common.bash

    exec {progress_fd}> \
         >( gawk '
...
        sudo find /                                 \
...
            | tee /dev/fd/$progress_fd              \
...
    exec {progress_fd}<&-

hope to help : ]

all my content is published under ZeroBSD license = zero restrictions + zero warranty

GetPackageOriginalFile sudo /etc/sudoers): No such file or directory

This is on a freshly installed system.

::: Sourcing /home/jfchevrette/.config/aconfmgr/30-base.sh...
:::: Downloading package sudo (sudo-1.8.30-1-x86_64.pkg.tar.xz) to pacman's cache
::::: Proceed? [Y/n]


Package (1)  Old Version  New Version  Net Change

core/sudo    1.8.30-1     1.8.30-1       0.00 MiB

Total Download Size:  0.00 MiB

:: Proceed with download? [Y/n]
(1/1) checking keys in keyring                                                                                                 [-----------------------------------------------------------------------------] 100%
(1/1) checking package integrity                                                                                               [-----------------------------------------------------------------------------] 100%
::::: Done.
:::: Unable to find package file for package sudo!
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1379 [AconfGetPackageOriginalFile]
::::: /usr/lib/aconfmgr/helpers.bash:185 [GetPackageOriginalFile]
::::: /home/jfchevrette/.config/aconfmgr/30-base.sh:37 [source]
::::: /usr/lib/aconfmgr/common.bash:112 [AconfCompileOutput]
::::: /usr/lib/aconfmgr/common.bash:779 [AconfCompile]
::::: /usr/lib/aconfmgr/apply.bash:168 [AconfApply]
::::: /usr/lib/aconfmgr/main.bash:163 [Main]
::::: /usr/lib/aconfmgr/main.bash:177 [source]
::::: /usr/bin/aconfmgr:26 [main]
/home/jfchevrette/.config/aconfmgr/30-base.sh: line 37: $(GetPackageOriginalFile sudo /etc/sudoers): No such file or directory
:::: Fatal error! Stack trace:
::::: /home/jfchevrette/.config/aconfmgr/30-base.sh:37 [source]
::::: /usr/lib/aconfmgr/common.bash:112 [AconfCompileOutput]
::::: /usr/lib/aconfmgr/common.bash:779 [AconfCompile]
::::: /usr/lib/aconfmgr/apply.bash:168 [AconfApply]
::::: /usr/lib/aconfmgr/main.bash:163 [Main]
::::: /usr/lib/aconfmgr/main.bash:177 [source]
::::: /usr/bin/aconfmgr:26 [main]

When I look in my local pacman cache, I see a file with a different extension for the sudo package: /var/cache/pacman/pkg/sudo-1.8.30-1-x86_64.pkg.tar.zst. Could this be why I'm having this error?

Add sudo server or loop

aconfmgr fails due to sudo timeouts. This is particularly annoying as it happens mid run if ran as a non root user.

Add support for aurutils

Please add aur/aurutils support as yay and a bunch are already supported.

AddPackageGroup

On the first run, I've noticed that I have quite a few AddPackage entries that belong to base and base-devel. I believe it would be a quality of life improvement if there were an AddPackageGroup. For now, I just dumped base and base-devel as a list of ignored packages (I'm installing them as a part of the pacstrap command anyway).

In the future, maybe there could also be automatic group detection when doing a save. Until then, maybe save could just add this to 99-unsorted:

# There's a high chance you may want to uncomment these:

# AddPackageGroup base
# AddPackageGroup base-devel

I understand that groups can change, which kills the idempotence. Some package you depend on may get excluded, and you'll be left with a broken script or something of sorts. However, IMO, the added brevity/clarity is appealing, and aconfmgr isn't perfectly idempotent as-is anyway, since without version pinning, something that's saved today may get applied as something completely else tomorrow (say, a major update of some package provides a different set of executables). There could also be a warning printed, when some package disappears from a group, after doing an apply (that requires some state though, to keep track of what's in each group).

Reached 1000 lost files while in directory that contains 29 files.

Running aconfmgr save gave the warning:

Warning: reached 1000 lost files while in directory /etc/systemd/system.

But if I list the contents (with ls -ar, find, or tree) I get

/etc/systemd/system
/etc/systemd/system/getty.target.wants
/etc/systemd/system/getty.target.wants/[email protected]
/etc/systemd/system/multi-user.target.wants
/etc/systemd/system/multi-user.target.wants/remote-fs.target
/etc/systemd/system/multi-user.target.wants/org.cups.cupsd.path
/etc/systemd/system/multi-user.target.wants/avahi-daemon.service
/etc/systemd/system/multi-user.target.wants/NetworkManager.service
/etc/systemd/system/multi-user.target.wants/tlp.service
/etc/systemd/system/multi-user.target.wants/gpm.service
/etc/systemd/system/multi-user.target.wants/earlyoom.service
/etc/systemd/system/printer.target.wants
/etc/systemd/system/printer.target.wants/org.cups.cupsd.service
/etc/systemd/system/sockets.target.wants
/etc/systemd/system/sockets.target.wants/org.cups.cupsd.socket
/etc/systemd/system/sockets.target.wants/avahi-daemon.socket
/etc/systemd/system/sockets.target.wants/sshd.socket
/etc/systemd/system/dbus-org.freedesktop.Avahi.service
/etc/systemd/system/tlp.service.d
/etc/systemd/system/tlp.service.d/override.conf
/etc/systemd/system/systemd-rfkill.service
/etc/systemd/system/systemd-rfkill.socket
/etc/systemd/system/dbus-org.freedesktop.NetworkManager.service
/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service
/etc/systemd/system/timers.target.wants
/etc/systemd/system/timers.target.wants/snapper-timeline.timer
/etc/systemd/system/timers.target.wants/snapper-cleanup.timer
/etc/systemd/system/sleep.target.wants
/etc/systemd/system/sleep.target.wants/tlp-sleep.service
/etc/systemd/system/sysinit.target.wants
/etc/systemd/system/sysinit.target.wants/systemd-timesyncd.service

That's nowhere near 1000 files.

Delete aconfmgr directory from /tmp when exiting

The /tmp/aconfmgt- directory doesn't get deleted when the program exits. Depending on the system, this can take up a fair amount of room (especially on the first run before all the necessary ignores are added).

Sync package DB after updating pacman.conf

I was using aconfmgr to apply my system configuration to another machine and ran into the problem that a package from another repository couldn't be found. This occurs because I also do a CopyFile /etc/pacman.conf, which happens right before in the "Installing priority files" step and adds a new repository. But the package database is not updated afterwards, so that pacman cannot find the package.

I fixed the problem by doing a pacman --sync --refresh after the installation of the priority files. If you like, I can create a PR.

[Feature Request] Script whitelist.

Each created sh-file can be seen as a module.
For example I could have a sh-file that sets up lmms as well as some plugins and other stuff related to music production.
So this is my music module, there might be modules for development in various languages and so on.

So now I have a collection of modules but for a workstation I might just want to setup a certain combination of modules to use on this system.
My computer for work might only use the modules for programming languages used at work.
While my computer at home uses a different set of modules including the one for music production.

I'm aware that this can be done with branches. But having a branch for each setup seems like a difficult thing, especially when the number of different used combinations increases.
And using a module system allows you to easily turn on and off modules when needed.

So my suggestion would be to allow a whitelist that is stored on each system and determines which scripts are active and considered by aconfmgr (all others will be ignored). The file's path could be an optional parameter to aconfmgr apply and save.

Add `AddPath`, which would un-ignore a subdirectory of an ignored directory

Add a new parameter to the configuration, that would enable adding select paths inside overall ignored ones.

For example,
To be able to backup /usr/lib/aconfmgr, while ignoring everything else, like:

IgnorePath '/usr/lib'
AddPath '/usr/lib/aconfmgr'

Something similar to how gitignore file works

Paranoid mode with priority files does not work

If I have the following in my config, and the file in my repo has changes to be applied (different from the one on my system)

CopyFile /etc/pacman.conf

And I run aconfmgr with the --paranoid option, it will ask me if I want to install the file, and then ask about overwriting changed files, to which if I respond with d to see a diff, nothing is shown

: Applying configuration...
:: Installing priority files...
::: Installing /etc/pacman.conf...
:::: Proceed? [Y/n] y
:::: Done.
::: Done.
:: Configuring packages...
::: Done.
:: Configuring files...
::: Overwriting 1 changed files.
:::: Proceed? [Y/n/d] d
:::: Overwriting the following changed files:
* /etc/pacman.conf
:::: Proceed? [Y/n/d] y
:::: Overwriting /etc/pacman.conf...
::::: Proceed? [Y/n/d] d
::::: Proceed? [Y/n/d] d
::::: Proceed? [Y/n/d] d
::::: Proceed? [Y/n/d] d

The problem appears to be that the file is modified after responding yes the first time, which then makes diff return no differences. Moreover the fact the file has already been applied makes the second prompt about overwriting the file pointless.

Non-priority files work fine as expected and a diff is shown.

Running save without any parameters defaults to reading configuration from /etc/xdg/aconfmgr

As per title, despite what the README says, aconfmgr does not seem to default to ~/.config/aconfmgr, but rather to /etc/xdg/aconfmgr. When the folder does not exist (and it doesnt at first), it exits complaining that permission is denied for creating the folder.

Unknown paccheck output line: unknown option: 'SyncFirst'

During the initial aconfmgr save, the following line is printed to the console:

:::: Unknown paccheck output line: warning:\ config\ /etc/pacman.conf\ line\ 20:\ unknown\ option\ \'SyncFirst\'

Not sure whether this is an issue or not.

"AddPackageGroup base" results in missing package linux-firmware

I am writing a config that starts with

AddPackageGroup base
AddPackageGroup base-devel

When running aconfmgr save it says

: Saving configuration...
:: Examining packages...
::: Found 1 missing packages. Un-registering.
:::: Done.
::: Done.

and I find RemovePackage linux-firmware in 99-unsorted.sh under Missing packages. Yet this should not be the case, since linux-firmware is both installed on my system (confirmed pacman -Qi linux-firmware) and in the package group base (is listed in pacman -Sqg base)

If I comment out AddPackageGroup base then run aconfmgr save, 99-unsorted.sh contains the following:

# Thu Jan  3 22:08:16 PST 2019 - Unknown packages


AddPackage bash # The GNU Bourne Again shell
AddPackage bzip2 # A high-quality data compression program
AddPackage coreutils # The basic file, shell and text manipulation utilities of the GNU operating system
AddPackage cryptsetup # Userspace setup tool for transparent encryption of block devices using dm-crypt
AddPackage device-mapper # Device mapper userspace library and tools
AddPackage dhcpcd # RFC2131 compliant DHCP client daemon
AddPackage diffutils # Utility programs used for creating patch files
AddPackage e2fsprogs # Ext2/3/4 filesystem utilities
AddPackage filesystem # Base Arch Linux files
AddPackage gcc-libs # Runtime libraries shipped by GCC
AddPackage glibc # GNU C Library
AddPackage inetutils # A collection of common network programs
AddPackage iproute2 # IP Routing Utilities
AddPackage iputils # Network monitoring tools, including ping
AddPackage jfsutils # JFS filesystem utilities
AddPackage less # A terminal based program for viewing text files
AddPackage licenses # Standard licenses distribution package
AddPackage linux # The Linux kernel and modules
AddPackage logrotate # Rotates system logs automatically
AddPackage lvm2 # Logical Volume Manager 2 utilities
AddPackage man-db # A utility for reading man pages
AddPackage man-pages # Linux man pages
AddPackage mdadm # A tool for managing/monitoring Linux md device arrays, also known as Software RAID
AddPackage nano # Pico editor clone with enhancements
AddPackage netctl # Profile based systemd network management
AddPackage perl # A highly capable, feature-rich programming language
AddPackage procps-ng # Utilities for monitoring your system and its processes
AddPackage psmisc # Miscellaneous procfs tools
AddPackage reiserfsprogs # Reiserfs utilities
AddPackage s-nail # Mail processing system with a command syntax reminiscent of ed
AddPackage shadow # Password and account management tool suite with support for shadow files and PAM
AddPackage sysfsutils # System Utilities Based on Sysfs
AddPackage systemd-sysvcompat # sysvinit compat for systemd
AddPackage tar # Utility used to store, backup, and transport files
AddPackage usbutils # USB Device Utilities
AddPackage vi # The original ex/vi text editor
AddPackage xfsprogs # XFS filesystem utilities

and there is no problem (linux-firmware is a required package of linux)

aconfmgr cannot remove directory from backup?

Hi,

running into a strange issue here:
I deleted /boot/grub after I switched to systemd-boot.
Afterwards, aconfmgr did not work anymore due to this:

[foo@bar boot]$ /usr/bin/aconfmgr --color never save
: Collecting data...
:: Compiling user configuration...
::: Using configuration in /home/foo/.config/aconfmgr
::: Sourcing /home/foo/.config/aconfmgr/10-ignores.sh...
::: Sourcing /home/foo/.config/aconfmgr/99-unsorted.sh...
rm: das Entfernen von '/tmp/aconfmgr-foo/output/files//boot/grub' ist nicht möglich: Ist ein Verzeichnis
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/helpers.bash:236 [RemoveFile]
::::: /home/foo/.config/aconfmgr/99-unsorted.sh:2860 [source]
::::: /usr/lib/aconfmgr/common.bash:100 [AconfCompileOutput]
::::: /usr/lib/aconfmgr/common.bash:713 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/lib/aconfmgr/main.bash:152 [Main]
::::: /usr/lib/aconfmgr/main.bash:169 [source]
::::: /usr/bin/aconfmgr:26 [main]

sorry for the German but it basically says that it cannot delete /tmp/aconfmgr-foo/output/files//boot/grub because it is a folder.

Did I do something wrong? Is deleting folders not supported in aconfmgr?

Allow to manage revision of foreign packages

Thank you for the nice project. It has definitively helped to keep my Arch system clean as far as possible.

Recently I've started to use aconfmgr on two systems, and sure enough it will install all my configured packages on both hosts. However, for the Foreign packages, this leaves a bit of an uneasy feeling: They do not receive the same level of vetting as a package of an TU in the official repositories recieves. And even if I trust a specific author, I will not get notified when the package maintainer on the AUR changes, etc.pp. creating a trust/security issue when aconfmgr automatically installs the latest version. So it would be nice to be able to pin foreign packages to a version that I have manually reviewed on any of my systems.

As the AUR uses git nowadays, it might be as easy as including the respective git revision alongside the package within the installation line.

Caveat:
Looking at the code, though, while this should be easily doable with the default, manual makepkg-based installation routine, it will not work for the third party-managers that easily.

I know that it's more in the projects spirit to move such enhancements to custom helpers, even though I personally might file it more as a 'security enhancement', which might be of interest. But maybe, even if this is not integrated into the main helpers, maybe there will be some interesting insights into an implementation. A potential implementation idea I have revolves around tracking the pkgbuild-repos as submodules and building from there. That way, they could be updated with a recursive git-pull, reviewed with a diff, and meanwhile we would guarantee a consistent system state with regards to AUR-package versions.

Thank you for your consideration.

Document high-level 'how it works' / is a new install and immediate `save` necessary

I was trying to figure out how this works, and in particular if it needed to store somewhere a checksum of all files on the system, and so should really be first run on a fresh install.

From looking at the code, it seems not, because it can grab a file list and MD5s from pacman. Nice. But does this mean there are some files that might concievably be changed, but aren't captured in any packages' listing? Or is base (and in particular it looks like filesystem) a complete listing?

Relatedly, I noticed a PR for upgrading base dependency - is it always required to keep base and aconfmgr in sync, i.e. does every release of base necessitate an aconfmgr release?

Add `IgnorePath` defaults for new configurations/packages

Hey Cybershadow,

This is basically a request for sane defaults or documentation on a default suggested IgnorePath list.

For example it is safe to assume that any Arch user will want to ignore /var/log from the scans and I can't imagine anyone wanting to include the pacman cache either.

I have only just started to test aconfmgr but my list is already 21 lines long and I don't think I have excluded everything that needs excluding as yet (it is still scanning).

Thanks,

Alan Jenkins

custom ignore_paths

I have some NFS volumes mounted outside of /mnt and /media. It would be great if there was a command line argument to add to ignore_paths so that those folders could be skipped when running aconfmgr save.

Thanks!

Support for journald

Hello,

I want to run aconfmgr as a systemd service and have its output put into my journal.
Unfortunate, this does not seem to work atm.

You can create a service file to start aconfmgr but the journal will only show
Feb 08 17:13:14 Mika aconfmgr[29198]: [1.2K blob data]
instead of the real output.
I already tried the --color Never option, does not help.

Thanks in advance!

Unknown paccheck output line: mtree data not available

During the initial aconfmgr save, the following lines are printed to the console:

:::: Unknown paccheck output line: warning:\ lsb-release:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)
:::: Unknown paccheck output line: warning:\ lsb-release:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)
:::: Unknown paccheck output line: warning:\ manjaro-alsa:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)
:::: Unknown paccheck output line: warning:\ manjaro-alsa:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)
:::: Unknown paccheck output line: warning:\ manjaro-pulse:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)
:::: Unknown paccheck output line: warning:\ manjaro-pulse:\ mtree\ data\ not\ available\ \(No\ such\ file\ or\ directory\)

Not sure whether these are issues or not.

No such file or directory when running aconfmgr save

$ ./aconfmgr save
: Collecting data...
:: Compiling user configuration...
::: Using configuration in ./config
::: Done (configuration not found).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating managed files...
:::: Done.
::: Searching for lost files...
:::: Warning: reached 1000 lost files while in directory /usr/share/man/man3.
::::: Perhaps add IgnorePath /usr/share/man/man3/\* (or a parent directory) to configuration to ignore it.
:::: Done (78530 lost files).
::: Searching for modified files...
:::: Installing foreign dependency pacutils:
::::: Detecting AUR helper...
:::::: pacaur... No
:::::: yaourt... No
:::::: makepkg... Yes
::::: Building foreign package pacutils from source.
:::::: Making sure the base-devel group is installed...
::::::: Done.
:::::: Cloning...
Cloning into 'pacutils'...
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 13 (delta 1), reused 13 (delta 1)
Unpacking objects: 100% (13/13), done.
Checking connectivity... done.
::::::: Done.
:::::: Checking dependencies...
::::::: Done.
:::::: Building...
src/common.bash: line 668: HOME=/home/harleypig/projects/aconfmgr/tmp/aur/pacutils/home: No such file or directory
::::::: Fatal error! Stack trace:
:::::::: src/common.bash:1 [AconfMakePkg]
:::::::: src/common.bash:702 [AconfInstallForeign]
:::::::: src/common.bash:723 [AconfNeedProgram]
:::::::: src/common.bash:199 [AconfCompileSystem]
:::::::: src/common.bash:451 [AconfCompile]
:::::::: src/save.bash:9 [AconfSave]
:::::::: ./aconfmgr:164 [Main]
:::::::: ./aconfmgr:178 [main]

The offending path exists:

$ ls -Al /home/harleypig/projects/aconfmgr/tmp/aur/pacutils/home
total 0

I tried tracing through it, but I'm quite picking up on your way of doing things yet ...

Sort 99-unsorted.sh by repo?

By default the packages are in a big long alphabetical list. I think it would be easier to parse and rewrite if the list was separated into core/ extra/ community/ aur/ etc.

I get a broken pipe error when I run aconfmgr save

I included all the text in case something useful is in there. You can just skip to the bottom if you want to:

aconfmgr save
: Collecting data...
:: Compiling user configuration...
::: Using configuration in /home/luke/.config/aconfmgr
::: Sourcing /home/luke/.config/aconfmgr/99-unsorted.sh...
::: Done (259 native packages, 8 foreign packages, 7490 files).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating managed files...
:::: Done.
::: Searching for lost files...
[sudo] password for luke: 
:::: Warning: reached 1000 lost files while in directory /usr/share/backgrounds/gnome.
::::: Perhaps add IgnorePath /usr/share/backgrounds/gnome/\* (or a parent directory) to configuration to ignore it.
:::: Done (23643 lost files).
::: Searching for modified files...
:::: android-sdk: /opt/android-sdk/tools/NOTICE.txt
:::: android-sdk: /opt/android-sdk/tools/emulator
:::: android-sdk: /opt/android-sdk/tools/emulator64-arm
:::: android-sdk: /opt/android-sdk/tools/emulator64-mips
:::: android-sdk: /opt/android-sdk/tools/emulator64-x86
:::: android-sdk: /opt/android-sdk/tools/lint
:::: android-sdk: /opt/android-sdk/tools/mksdcard
:::: android-sdk: /opt/android-sdk/tools/source.properties
:::: android-sdk: /opt/android-sdk/tools/ant/build.xml
:::: android-sdk: /opt/android-sdk/tools/apps/SdkController/assets/intro_help.html
:::: android-sdk: /opt/android-sdk/tools/apps/SdkController/res/layout/sensors.xml
:::: android-sdk: /opt/android-sdk/tools/lib/ant-tasks.jar
:::: android-sdk: /opt/android-sdk/tools/lib/archquery.jar
:::: android-sdk: /opt/android-sdk/tools/lib/asset-studio.jar
:::: android-sdk: /opt/android-sdk/tools/lib/builder-model.jar
:::: android-sdk: /opt/android-sdk/tools/lib/chimpchat.jar
:::: android-sdk: /opt/android-sdk/tools/lib/common.jar
:::: android-sdk: /opt/android-sdk/tools/lib/ddmlib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/ddms.jar
:::: android-sdk: /opt/android-sdk/tools/lib/ddmuilib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/draw9patch.jar
:::: android-sdk: /opt/android-sdk/tools/lib/dvlib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/emma_device.jar
:::: android-sdk: /opt/android-sdk/tools/lib/fat32lib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/hardware-properties.ini
:::: android-sdk: /opt/android-sdk/tools/lib/hierarchyviewer2.jar
:::: android-sdk: /opt/android-sdk/tools/lib/hierarchyviewer2lib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/jobb.jar
:::: android-sdk: /opt/android-sdk/tools/lib/layoutlib-api.jar
:::: android-sdk: /opt/android-sdk/tools/lib/lint-api.jar
:::: android-sdk: /opt/android-sdk/tools/lib/lint-checks.jar
:::: android-sdk: /opt/android-sdk/tools/lib/lint.jar
:::: android-sdk: /opt/android-sdk/tools/lib/manifest-merger.jar
:::: android-sdk: /opt/android-sdk/tools/lib/monkeyrunner.jar
:::: android-sdk: /opt/android-sdk/tools/lib/ninepatch.jar
:::: android-sdk: /opt/android-sdk/tools/lib/plugin.prop
:::: android-sdk: /opt/android-sdk/tools/lib/rule-api.jar
:::: android-sdk: /opt/android-sdk/tools/lib/screenshot2.jar
:::: android-sdk: /opt/android-sdk/tools/lib/sdk-common.jar
:::: android-sdk: /opt/android-sdk/tools/lib/sdklib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/sdkmanager.jar
:::: android-sdk: /opt/android-sdk/tools/lib/sdkstats.jar
:::: android-sdk: /opt/android-sdk/tools/lib/sdkuilib.jar
:::: android-sdk: /opt/android-sdk/tools/lib/swtmenubar.jar
:::: android-sdk: /opt/android-sdk/tools/lib/traceview.jar
:::: android-sdk: /opt/android-sdk/tools/lib/uiautomatorviewer.jar
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86/libcairo-swt.so
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86/monitor
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86/configuration/config.ini
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86/plugins/com.ibm.icu_4.4.2.v20110823.jar
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86_64/libcairo-swt.so
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86_64/monitor
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86_64/configuration/config.ini
:::: android-sdk: /opt/android-sdk/tools/lib/monitor-x86_64/plugins/com.ibm.icu_4.4.2.v20110823.jar
:::: android-sdk: /opt/android-sdk/tools/lib/pc-bios/bios.bin
:::: android-sdk: /opt/android-sdk/tools/lib/pc-bios/vgabios-cirrus.bin
:::: android-sdk: /opt/android-sdk/tools/proguard/proguard-android-optimize.txt
:::: android-sdk: /opt/android-sdk/tools/proguard/proguard-android.txt
:::: android-sdk: /opt/android-sdk/tools/support/annotations.jar
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/root/res/menu/main.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/root/res/values/dimens.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/root/res/values/strings.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/BlankActivity/root/src/app_package/SimpleActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/root/res/layout/activity_fullscreen.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/FullscreenActivity/root/src/app_package/FullscreenActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/root/res/layout/activity_login.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/root/res/values/strings.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/LoginActivity/root/src/app_package/LoginActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/res/layout/activity_content_detail.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/res/layout/activity_content_list.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/res/layout/activity_content_twopane.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/res/layout/fragment_content_detail.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/src/app_package/ContentDetailActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/src/app_package/ContentDetailFragment.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/src/app_package/ContentListActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/MasterDetailFlow/root/src/app_package/ContentListFragment.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/SettingsActivity/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/SettingsActivity/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/SettingsActivity/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/activities/SettingsActivity/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/activities/SettingsActivity/root/src/app_package/SettingsActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/gradle/wrapper/gradle/wrapper/gradle-wrapper.properties
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/root/src/app_package/AppWidget.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/AppWidget/root/src/app_package/AppWidgetConfigureActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/template_blank_fragment.png
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/root/res/layout/fragment_blank.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BlankFragment/root/src/app_package/BlankFragment.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BroadcastReceiver/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/BroadcastReceiver/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/ContentProvider/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/ContentProvider/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/CustomView/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/CustomView/root/src/app_package/CustomView.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Daydream/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Daydream/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/Daydream/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Daydream/root/src/app_package/DreamService.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Daydream/root/src/app_package/SettingsActivity.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/IntentService/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/IntentService/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/IntentService/root/AndroidManifest.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/root/res/layout/fragment_grid.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/root/res/layout/fragment_list.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/ListFragment/root/src/app_package/ListFragment.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Notification/recipe.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Notification/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/Notification/root/src/app_package/NotificationHelper.java.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Service/globals.xml.ftl
:::: android-sdk: /opt/android-sdk/tools/templates/other/Service/template.xml
:::: android-sdk: /opt/android-sdk/tools/templates/other/Service/root/AndroidManifest.xml.ftl
:::: bash: /etc/skel/.bashrc
:::: cups: /etc/cups/subscriptions.conf
:::: filesystem: /etc/arch-release
:::: filesystem: /etc/crypttab
:::: filesystem: /etc/fstab
:::: filesystem: /etc/group
:::: filesystem: /etc/gshadow
:::: filesystem: /etc/hosts
:::: filesystem: /etc/passwd
:::: filesystem: /etc/profile
:::: filesystem: /etc/resolv.conf
:::: filesystem: /etc/shadow
:::: filesystem: /etc/shells
:::: filesystem: /usr/lib/os-release
:::: gdm: /etc/gdm/Xsession
:::: gdm: /etc/gdm/custom.conf
:::: glibc: /etc/locale.gen
:::: gnome-backgrounds: /usr/share/gnome-background-properties/gnome-backgrounds.xml
:::: gnome-system-log: /usr/share/help/ja/gnome-system-log/pref-font-size.page
:::: grub: /boot/grub/grub.cfg
:::: grub: /etc/default/grub
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.alias
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.alias.bin
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.dep
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.dep.bin
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.symbols
:::: linux: /usr/lib/modules/4.7.4-1-ARCH/modules.symbols.bin
:::: mkinitcpio: /etc/mkinitcpio.conf
:::: pacman: /etc/pacman.conf
:::: pacman-mirrorlist: /etc/pacman.d/mirrorlist
:::: pam: /etc/environment
:::: playonlinux: /usr/share/playonlinux/etc/playonlinux.png
:::: playonlinux: /usr/share/playonlinux/etc/playonlinux15.png
:::: playonlinux: /usr/share/playonlinux/etc/playonlinux16.png
:::: playonlinux: /usr/share/playonlinux/etc/playonlinux22.png
:::: playonlinux: /usr/share/playonlinux/etc/playonlinux32.png
:::: shadow: /etc/default/useradd
:::: systemd: /etc/systemd/journald.conf
:::: tlp: /etc/default/tlp
:::: vlc: /usr/lib/vlc/plugins/plugins.dat
:::: Done (176 modified files).
::: Reading file attributes...
:::: Reading file types...
[sudo] password for luke: 
/usr/lib/aconfmgr/common.bash: line 929: printf: write error: Broken pipe
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1 [AconfCompileSystem]
::::: /usr/lib/aconfmgr/common.bash:451 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/bin/aconfmgr:164 [Main]
::::: /usr/bin/aconfmgr:178 [main]

I'm not sure if it's relevant, but i use fish as my shell.

use multiple file directory and be able to set them

So let's say i have many computer that share configuration but also have specific things to each one of them
wouldn't it be easier to have
files-laptop/ #files specific to laptop
files-desktop/ #files specific to desktop
files-server/ #files specific to server
files-shared/ #files shared between all of them
etc...
?
well being able to set it up that way would be nice ^

"cp: cannot create regular file..." during Registering files phase

[Question] Why is /home ignored by default?

Things like my .zshrc and similar files in /home/username are the configurations files that are the most important to me.
Why are they ignored by default? Can this behaviour be changed?

[Feature Request] Introduce hooks

Some setup actions need to perfromed after a package has been installed or after specific files have already been copied over.

For example if you want to use a specific shell (zsh) you have to install the package and copy over all necessary configuration files (.zshrc, ...).
Then you have to set the shell as your default shell. This can only be done after the package has already been installed.

So we need a way to specify scripts that are run after aconfmgr apply.
So basically a function PostApply which runs given code or script file after aconfmgr apply has been executed.

Error if AddPackage arg doesn't exist

I've noticed that if I typo the argument to AddPackage, or don't supply --foreign when it's only in the AUR, that it's just silently ignored, as if I hadn't specified it at all.

Perhaps it would be better to fail, or at least warn?

[feature request] Add an optional configuration file, to store arguments that can be specified on the command line

As of now this may be low priority, due to limited amount of CLI options.

However, it would be beneficial to have a aconfmgr.conf or config file ~~inside the configuration directory~~ (most likely outside of it, if it's a custom path), that would hold custom CLI arguments, like aur-helper of choice, paranoid or verbose state, or custom configuration path

Support `yay` AUR helper

Hi! Any chance of adding yay to the list of supported AUR helpers?

I believe that, in terms of the commands used by aconfmgr, using yay should be very similar, if not equal, to using pacaur.

Once again, thank you for your work!

base_devel_missing: unbound variable

Hi, I'm trying to backup my arch Linux config. However, I have ran into some errors while doing so.

I get the error: :::::: Making sure the base-devel group is installed...
src/common.bash: line 805: base_devel_missing: unbound variable
::::::: Fatal error! Stack trace:

I am not sure what is wrong. I had a look at common.bash where the supposed unbound variable is, but I couldn't see any errors.

Also, may I ask, what are all the "missing files", and should I ignore these in 10-ignores.sh?

This program saves just my configurations files made in arch Linux but not any other files right?

Thanks.

: Collecting data...
:: Compiling user configuration...
::: Using configuration in ./config
::: Sourcing ./config/10-ignore.sh...
::: Done (0 native packages, 0 foreign packages, 0 files).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating managed files...
:::: Done.
::: Searching for lost files...
:::: Warning: reached 1000 lost files while in directory /boot/efi/EFI/Microsoft/Boot/et-EE.
::::: Perhaps add IgnorePath /boot/efi/EFI/Microsoft/Boot/et-EE/* (or a parent directory) to configuration to ignore it.
:::: Scanning /usr/lib/python2.7/site-packages/twisted.:::: Done (2955 lost files).
::: Cleaning up ignored files' directories...
:::: Done.
::: Searching for modified files...
:::: Installing foreign dependency pacutils:
::::: Detecting AUR helper...
:::::: aurman... No
:::::: pacaur... No
:::::: yaourt... No
:::::: makepkg... Yes
::::: Building foreign package pacutils from source.
:::::: Using directory ./tmp/aur/pacutils.
:::::: Making sure the base-devel group is installed...
src/common.bash: line 805: base_devel_missing: unbound variable
::::::: Fatal error! Stack trace:
:::::::: src/common.bash:1 [AconfMakePkg]
:::::::: src/common.bash:1046 [AconfInstallForeign]
:::::::: src/common.bash:1067 [AconfNeedProgram]
:::::::: src/common.bash:326 [AconfCompileSystem]
:::::::: src/common.bash:718 [AconfCompile]
:::::::: src/save.bash:9 [AconfSave]
:::::::: src/main.bash:156 [Main]
:::::::: src/main.bash:173 [source]
:::::::: ./aconfmgr:26 [main]

code snippet for creating patches of modified config files

Thought I would share this code snippet I came up with to generate a 98-patches.sh. The idea is similar to inlining file edits, where sometimes if 1 out of 1000 lines changed, it may be handy to use a patch.

First, do aconfmgr save to get a 99-unsorted.sh. Then run this python script to check all lines with a "CopyFile" to see if the file belongs to a package, and if so create a patch vs the original, then add a corresponding line to 98-patches.sh

import os
import subprocess


def file_belongs_to_package(path):
    try:
        output = subprocess.check_output("pacman -Qo " + path, shell=True).decode('ascii')
    except: # if 'is owned by' not in output:
        return None, None
    # example of expected output:
    # /etc/pacman.conf is owned by pacman 5.1.3-1
    pkg_name, pkg_ver = output.strip().split(' ')[-2:]
    return pkg_name, pkg_ver


def diff_against_pkg_default(path, pkg_name, pkg_ver):
    diff_file = pkg_name + '_' + os.path.split(path)[1] + '.diff'
    # https://bbs.archlinux.org/viewtopic.php?pid=1464430#p1464430
    # Example:
    # tar -xOf /var/cache/pacman/pkg/pacman-5.1.3-1-x86_64.pkg.tar.xz etc/pacman.conf | diff -u - /etc/pacman.conf
    command = f'tar -xOf /var/cache/pacman/pkg/{pkg_name}-{pkg_ver}-*.pkg.tar.xz {path[1:]} | diff -u - {path} > {diff_file}'
    subprocess.call(command, shell=True)
    return diff_file


with open('99-unsorted.sh', 'r') as f:
    lines = f.read(-1).splitlines()

aconf_cmds = []
for line in lines:
    if not line.startswith('CopyFile'):
        continue
    path = line.split(' ')[1]
    pkg_name, pkg_ver = file_belongs_to_package(path)
    if pkg_name is None:
        continue
    diff_file = diff_against_pkg_default(path, pkg_name, pkg_ver)
    aconf_cmd = f'patch "$(GetPackageOriginalFile {pkg_name} {path})" ./config/{diff_file}'
    aconf_cmds.append(aconf_cmd)

with open ('98-patches.sh', 'w') as f:
    f.write('\n'.join(aconf_cmds))

sample output:

patch "$(GetPackageOriginalFile bluez /etc/bluetooth/main.conf)" ./config/bluez_main.conf.diff
patch "$(GetPackageOriginalFile filesystem /etc/fstab)" ./config/filesystem_fstab.diff
patch "$(GetPackageOriginalFile filesystem /etc/group)" ./config/filesystem_group.diff
patch "$(GetPackageOriginalFile filesystem /etc/hosts)" ./config/filesystem_hosts.diff
patch "$(GetPackageOriginalFile glibc /etc/locale.gen)" ./config/glibc_locale.gen.diff
patch "$(GetPackageOriginalFile mkinitcpio /etc/mkinitcpio.conf)" ./config/mkinitcpio_mkinitcpio.conf.diff
patch "$(GetPackageOriginalFile pacman /etc/pacman.conf)" ./config/pacman_pacman.conf.diff
patch "$(GetPackageOriginalFile pacman-mirrorlist /etc/pacman.d/mirrorlist)" ./config/pacman-mirrorlist_mirrorlist.diff
patch "$(GetPackageOriginalFile filesystem /etc/passwd)" ./config/filesystem_passwd.diff
patch "$(GetPackageOriginalFile pulseaudio /etc/pulse/system.pa)" ./config/pulseaudio_system.pa.diff
patch "$(GetPackageOriginalFile filesystem /etc/shells)" ./config/filesystem_shells.diff
patch "$(GetPackageOriginalFile sudo /etc/sudoers)" ./config/sudo_sudoers.diff

At this point you could cherry pick one or two if using a patch makes sense, delete 99-unsorted.sh, and verify the respective "CopyFile" are gone after another aconfmgr save. Any thoughts or suggestions? Cheers !

Allow adding / installing packages with `--asdeps`

pacman allows you to install packages with the option --asdeps to mark them as dependencies.
This is useful for optional dependencies as pacman can then recognize them as orphans when the packages that optionally required them are removed.
Supporting this flag might also help with the limitations mentioned in the README.

Introduce a way to set file properties recursively (e.g. `SetFileProperty --recursive`)

The directory /boot is often formatted vfat. Thus, lots of files are detected having wrong permissions with a recommendation: SetFileProperty /boot/cmdline.txt mode 755
It would be nice to be able to mark the whole mount point /boot to disregard file permissions at all. Consider IgnorePermissionsPath '/boot/*'

Continue instead of failing when refusing to uninstall packages

Hello,

When using aconfmgr apply, I would like to have the option to refuse the uninstallation of some packages and still continue the process instead of aborting.

My use case is that I use yay to install aur packages and it installs build dependencies. I'd like to keep those dependencies around to speed up the update process, but aconfmgr always wants to remove then as they are not in my configuration.

I don't want to include these packages in my configuration because I don't use them for anything else, and it would get tedious to maintain this list (add new build dependencies but also remove obsolete ones as I update and remove aur packages)

Thanks in advance.

LC_ALL=C: command not found, printf: write error: Broken pipe

During a dry run aconfmgr (from the AUR) crashes and produces this error:

:::: Done (43 modified files).
::: Reading file attributes...
:::: Reading file types...
/usr/lib/aconfmgr/common.bash: line 928: LC_ALL=C: command not found
/usr/lib/aconfmgr/common.bash: line 928: printf: write error: Broken pipe
:::: Fatal error! Stack trace:
::::: /usr/lib/aconfmgr/common.bash:1 [AconfCompileSystem]
::::: /usr/lib/aconfmgr/common.bash:450 [AconfCompile]
::::: /usr/lib/aconfmgr/save.bash:9 [AconfSave]
::::: /usr/bin/aconfmgr:164 [Main]
::::: /usr/bin/aconfmgr:178 [main]

Tried this in both bash and zsh, output is the same. I don't know why LC_ALL is being interpreted as a command, rather than a variable.

Command used was: sudo aconfmgr save

Problem with read

I've traced this back to I believe line 793 of common.bash. I have two different issues, one with zsh and one with bash. This is what happens when I reach that point in the script, without hitting any keys; I'm running aconfmgr apply -c . and leaving it alone.

In bash, I get:

: Applying configuration...
:: Installing priority files...
::: Installing /etc/pacman.conf...
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: Proceed? [Y/n] 
:::: User abort

And in zsh I get:

: Applying configuration...
:: Installing priority files...
::: Installing /etc/pacman.conf...
:::: Proceed? [Y/n] ::::: Fatal error! Stack trace:
:::::: /usr/lib/aconfmgr/common.bash:1 [Confirm]
:::::: /usr/lib/aconfmgr/apply.bash:110 [AconfApply]
:::::: /usr/bin/aconfmgr:167 [Main]
:::::: /usr/bin/aconfmgr:178 [main]

From what I've been able to tell, other actions have worked fine, it's just copying over files that's an issue (removing files, and adding and removing packages has been fine).

"Skip" option in addition to d/y/n

It would be nice to have a 'skip' option in addition to the yes/no/display options in confirmation prompts.

grep dependency missing

grep is missing from PKGCONFIG, and also isn't installed at runtime like others. (Or perhaps it is, but it's used first.)

::: Querying package list...
src/common.bash: line 172: grep: command not found
:::: Fatal error! Stack trace:
::::: src/common.bash:172 [AconfCompileSystem]
::::: src/common.bash:783 [AconfCompile]
::::: src/save.bash:9 [AconfSave]
::::: src/main.bash:160 [Main]
::::: src/main.bash:177 [source]
::::: ./aconfmgr:26 [main]

Alternative aconfmgr directory?

My home partition is low on space and runs out of it when I run a save and files are copied to files/. I have space on my /var partition though. I could setup a symlink and set some permissions, but might there be a cleaner way to achieve this? Thanks!

Cleanup of files directory

The files directory never seems to be cleaned up, for instance if i "aconfmgr save", then ignore some files that were found and added to 99-unsorted.sh as "CopyFile ...", these files will stay in the files directory even though they are ignored.
Is this the expected behaviour and what could i do to only keep the files which actually exist in the configuration?

IgnorePath rule not working properly in select cases

IgnorePath '/boot' does not prevent /boot/grub/grub.cfg from being backed up. In the same way, '/usr/lib/modules' does not prevent kernel modules from being backed up.

IgnorePath '/boot' 
IgnorePath '/boot/*'

solves it, but Im not sure why only in those 2 cases the second line is necessary