hirak99 / yabsnap Goto Github PK
View Code? Open in Web Editor NEWBtrfs Scheduled Snapshot Manager for Arch
License: Apache License 2.0
Btrfs Scheduled Snapshot Manager for Arch
License: Apache License 2.0
Can't understand how yabsnap work with snapshots with trigger I
, witch are creates by pacman pre hook..
Config is:
$ grep -v "^#" /etc/yabsnap/configs/root.conf
[DEFAULT]
source = /
dest_prefix = /.snapshots/@root-
trigger_interval = 1 hour
min_keep_secs = 1800
keep_user = 1
keep_preinstall = 1
preinstall_interval = 5 minutes
keep_hourly = 5
keep_daily = 1
keep_weekly = 1
keep_monthly = 1
keep_yearly = 1
keep_preinstall
is 1
But list of snapshots are:
$ yabsnap list
Config: /etc/yabsnap/configs/root.conf (source=/)
Snaps at: /.snapshots/@root-...
20231024121406 I (6 days 23h ago) pacman --color=always --sync --needed p7zip
20231024123022 S (6 days 23h ago)
20231025092712 I (6 days 2h ago) pacman --color=always --sync --sysupgrade
20231025200741 I (5 days 15h ago) /usr/bin/pacman -U /home/nebulosa/Documents/yabsnap/yabsnap-2.0.7-2-any.pkg.tar.zst
20231026162630 I (4 days 19h ago) pacman --color=always --sync --sysupgrade
20231026230426 I (4 days 12h ago) pacman --color=always --sync --sysupgrade
20231027120119 I (3 days 23h ago) pacman --color=always --sync --sysupgrade
20231028112115 I (3 days ago) pacman --color=always --sync --sysupgrade
20231029142211 I (1 day 21h ago) pacman --color=always --sync --sysupgrade
20231030092423 I (1 day 2h ago) pacman --color=always --sync --sysupgrade
20231030120211 I (23h 32m ago) pacman --color=always --sync --sysupgrade
20231030193026 S (16h 4m ago)
20231030233958 I (11h 54m ago) pacman --color=always --sync --sysupgrade
20231031092224 S (2h 12m ago)
20231031094105 I (1h 53m ago) pacman --color=always --upgrade /home/nebulosa/.cache/pikaur/pkg/yabsnap-2.0.11-1-any.pkg.tar.zst
20231031103029 S (1h 4m ago)
20231031113006 S (4m 41s ago)
What did I miss?
As of AUR package version 1.05.3-1, yabsnap
has its systemd
unit files in /etc/systemd/system/
. However, units provided by installed packages should live in ‘/usr/lib/systemd/system/’:
Would it be possible to add a way to run a script after creating/removing a snapshot?
I have a cron job to run a fish script to populate a refind config with options to boot into snapshots.
It's this if you are curious: create-refind-fallbacks.txt
But it would be far better if it just ran after yabsnap made a modification to the root snapshots.
This could also allow me to have a script show a notification when it ran.
Some kind of implementation could look like this in the config:
post_transaction_script = /root/scripts/create-refind-fallbacks.fish
Or is there another way without having to implement it in yabsnap?
Thank you for your time and effort!
I can see that the install-to-dest.sh
script is grepping for Arch Linux
in the file /etc/issue
. I think that's a very fragile way of doing it since the system administrator may modify that file at any time (see issue(5)
for details).
Instead, make yabsnap
depend on the lsb-release
package and make sure (1) that /etc/lsb-release
exists, and (2) that $DISTRIB_ID
equals Arch
. Perhaps something along these lines:
ME="${0##*/}"
ETC_LSB_RELEASE='/etc/lsb-release'
test -r "$ETC_LSB_RELEASE" || {
printf '%s: cannot read %s\n' "$ME" "$ETC_LSB_RELEASE" >&2
exit 1
}
. "$ETC_LSB_RELEASE"
test "$DISTRIB_ID" = 'Arch' || {
printf '%s: Not an Arch based distro, not proceeding.\n' "$ME" >&2
exit 1
}
There should be a command to clear snapshots in bulk.
It should have options to specify which tags, and what algorithm should be applied to the clean up.
Abstract out the btrfs calls into a module, so that it is easier to create implementations for different snapshotting mechanisms.
I think I should report this issue on github because the PKGBUILD seems to be no problem.
I can't built with this error:
+ chmod -R g-w,a-w src/ /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/share/yabsnap
++ which selinuxenabled
+ mkdir -p /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin
+ ln -sf /usr/share/yabsnap/yabsnap.sh /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap
+ chmod 755 /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap
chmod: 无法对悬空符号链接 '/home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap' 进行操作==> 错误: 在 package() 中发生一个错误。 正在放弃...
错误: 未能构建 'yabsnap-2.0.2-1':
The ln -sf /usr/share/yabsnap/yabsnap.sh $PKGDIR/usr/bin/yabsnap
shold be ln -sf $PKGDIR/usr/share/yabsnap/yabsnap.sh $PKGDIR/usr/bin/yabsnap
or removing /
? There seems to be a missing point here and yabsnap-git in AUR has the same error.
sudo yabsnap create
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/share/yabsnap/code/main.py", line 189, in
main()
File "/usr/share/yabsnap/code/main.py", line 152, in main
if configs.is_schedule_enabled():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/yabsnap/code/configs.py", line 157, in is_schedule_enabled
for config in iterate_configs(None):
File "/usr/share/yabsnap/code/configs.py", line 138, in iterate_configs
config = Config.from_configfile(fname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/yabsnap/code/configs.py", line 98, in from_configfile
setattr(result, key, int(value))
^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '1jjjjjjjjjjjjjjjjjjjjjj'
Lines 137 to 143 in deb4d2e
The pacman snapshot gets taken a little too late and includes the /var/lib/pacman/db.lck
. It's a bit annoying to remove it every time you reset a pacman snapshot
The following exits successfully with no error -
yabsnap --verbose --source this_source_does_not_exist create
Need to add a descriptive error when the source is not found.
Forked from #11
Error, warnings, etc. should be colorized when appropriate so that they are harder to miss.
First of all, thanks for this project.
Secondly, there is a mistake in the instruction. Here:
https://github.com/hirak99/yabsnap/blob/master/README.md
In section "Recommended Subvolume Layout". If you follow the instruction step by step, the OS will not boot because it will not be able to mount "@.snapshots"
You probably need to change the last line of instructions or an entire section.
I tested this in Debian 12 (stable), but I think that the problem will also occur in other distros.
Source - https://barnettphd.com/snap-pac/configuration.html#environment-variables
snap-pac-hook has support for using SNAP_PAC_SKIP to skip snapper snapshot for pacman transaction.
Can you advise on this ->
I usually many time install and uninstall single application through pacman, this ends up creating snapshot which are barely worth it be snapshotted and ends up filling snapshots which will inturn delete important snapshots.
What do you to solve this ?
For me i had added SNAP_PAC_SKIP=yes in environment and when i do full system upgrade , i explicitly set SNAP_PAC_SKIP=no. But i have observed that i most of the time forget to do this. now i am in limbo, i either end up with all snapshot or end up with no snapshots for pacman.
I've tried reinstalling multiple times but I keep getting this error whenever I try to install or remove any packages. I've only had it happen on my laptop and not on my desktop despite both running the latest version of yabsnap:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/share/yabsnap/code/main.py", line 173, in
main()
File "/usr/share/yabsnap/code/main.py", line 167, in main
_config_operation(
File "/usr/share/yabsnap/code/main.py", line 114, in _config_operation
snapper.create(comment)
File "/usr/share/yabsnap/code/snap_operator.py", line 107, in create
self._create_and_maintain_n_backups(
File "/usr/share/yabsnap/code/snap_operator.py", line 89, in _create_and_maintain_n_backups
previous_snaps = [
File "/usr/share/yabsnap/code/snap_operator.py", line 89, in
previous_snaps = [
File "/usr/share/yabsnap/code/snap_operator.py", line 37, in _get_old_backups
yield snap_holder.Snapshot(pathname)
File "/usr/share/yabsnap/code/snap_holder.py", line 75, in init
self.metadata = _Metadata.load_file(self._metadata_fname)
File "/usr/share/yabsnap/code/snap_holder.py", line 65, in load_file
return cls(**json.load(f))
File "/usr/lib/python3.10/json/init.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Harden the CLI by validating a mount to be btrfs, before operations and on yabsnap list
and refuse operating if the mount is not btrfs.
I would really like to see snapshot comments made by yabsnap also show up in grub-btrfs boot menu. 'Type' doesn't really matter but would be a nice bonus. Not seeing the pacman command is the one thing keeping me from ditching snapper
I deleted Rollback folder inside /.snapshots
and rebooted, bootloader cant find my root partition now.
BTW,what is the rollback folder.
I do think about the possibility, that I could adopt this repo, to make it work for bcachefs.
Do you want a PR, or do you prefer me forking this project?
I do not know yet, how compatible the two implementations will be, as I have yet not looked how the bcachefs tooling API even looks like. But I could imagine, doing it both ways.
What do you prefer?
When you build/install yabsnap package it launch run_tests.sh and shows this warning:
/home/nebulosa/yabsnap/src/yabsnap-2.0.7/src/code/configs.py:177: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/nebulosa/yabsnap/src/yabsnap-2.0.7/src/code/example_config.conf' mode='r' encoding='UTF-8'>
for line in open(_example_config_fname()):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Created: /tmp/yabsnap_config_test_rdlhy2bi
............
----------------------------------------------------------------------
Ran 12 tests in 0.015s
OK
A little reseach shows me that you need edit file config.py like that:
with open(_example_config_fname()) as ecn:
for line in ecn:
line = line.strip()
if source and line.startswith("source ="):
line = f"source = {source}"
elif line.startswith("dest_prefix ="):
line = f"dest_prefix = /.snapshots/@{name}-"
lines.append(line)
and warning is gone.
Am I right? If so, please edit strings or I can open Pull Request for convenience
A very simple GIU would be a cool feature to improve usability and make Yabsnap an attractive alternative next to timeshift and snapper, since both already have a GUI. It shouldn't be anything wild or elaborate, but just offer the possibility to take snapshots manually, manage them and set the rhythm of how the snapshots should be taken.
Thanks for reading ^^
When you write yabsnap list
it shows all information about snapshopts, but when you want delete some of them it will be not so easy, because name of snapshots is 14-digit timestamp, that really hard to enter, only copy and past.
For convinience you can use the fzf in that scenario (e.g. bash alias snapctl
):
ls -d /.snapshots/*/ | sed 's/\/.snapshots\///g;s/\/$//g' | fzf -m --reverse --preview 'echo -n $(basename {1})": "; cat /.snapshots/{1}-meta.json | echo $(tr -d "\"}{")' --preview-window right:70%:wrap | xargs -I snap sudo yabsnap delete /.snapshots/snap'
It works as expected and already could be mentined in README, but really wants use yabsnap
with fzf
like that:
yabsnap list -q | fzf ... --preview yabsnap list {1} | xargs ... yabnsap delete ...
, where:
yabsnap list -q
it lists names of snapshots,
yabsnap list PATH|TIMESTAMP [--oneline]
shows info only for one snapshot (--oneline
- comma-separared info in one line)
Any thoughts?
First, thanks a lot for this tool! This is the best one I have found for managing automated snapshot.
Although it seems focused on system snapshots (and rollback), I would find it very useful to be able to manage snapshots for non-root users, i.e. created and owned by a user other than root
.
What I have tried so far:
$XDG_CONFIG_HOME/yabsnap/home.conf
with
source = /home/the_user
)dest_prefix = /home/the_user/.snapshots/@home-
)the_user
) yabsnap --source $XDG_CONFIG_HOME/yabsnap/home.conf create
.The next step would be to create a user systemd unit and timer to automate the snapshots.
My current issue is at step 2: yabsnap
exits successfully, with no output to neither stdout
nor stderr
, and no snapshot is created at the configured destination.
Am I doing something wrond? Is this use case not supported?
Thanks in advance for any advice!
In 262bceb you add the apache2 license header to the files.
This includes a copyright claim # Copyright 2022 Google LLC
.
Is this correct?
Also, in Contributing.md you set the requirement of signing Googles CLA to contribute.
Is this project owned by you personally, or by Google LLC
?
I have updated PKGBUILD using this article.
New PKGBUILD and the install file.
Why? Because, it's not quite right to build package through bash script - it cause errors in future like #17 and other mistakes with permission of files (e.g. .service files should be read only not executable)
I hope you find this usefil and will accept my edits
Hello ,
I am currently using yabsnap for managing snapshots on two volumes, namely @root
and @data.
The current functionality of yabsnap takes snapshots of both volumes whenever pacman performs any actions. However, I would like to propose the addition of a feature that allows users to toggle the behavior of yabsnap, specifically deciding whether snapshots should be taken for the @data
volume when pacman performs action. This toggle option would provide users with more control over the snapshot process, ensuring snapshots are only captured for the @root
volume when pacman performs actions.
Thanks
When executing pacman commands I get the following error:
(1/1) Triggering yabsnap pre-installation snapshots...
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/share/yabsnap/code/main.py", line 189, in <module>
main()
File "/usr/share/yabsnap/code/main.py", line 180, in main
_config_operation(
File "/usr/share/yabsnap/code/main.py", line 120, in _config_operation
snapper.on_pacman()
File "/usr/share/yabsnap/code/snap_operator.py", line 149, in on_pacman
self._create_and_maintain_n_backups(
File "/usr/share/yabsnap/code/snap_operator.py", line 122, in _create_and_maintain_n_backups
for expired in _all_but_last_k(previous_snaps, count - 1):
File "/usr/share/yabsnap/code/snap_operator.py", line 62, in _all_but_last_k
raise ValueError(f"k = {k} < 0")
ValueError: k = -1 < 0
error: command failed to execute correctly
error: failed to commit transaction (failed to run transaction hooks)
This is due to changes introduced with 12bc3b3 in the function _create_and_maintain_n_backups
.
When keep_preinstall is zero and one is substracted of count in the method _create_and_maintain_n_backups the function _all_but_last_k raises an exception.
My proposed fix is:
--- a/src/code/snap_operator.py
+++ b/src/code/snap_operator.py
@@ -122,1 +122,1 @@
- for expired in _all_but_last_k(previous_snaps, count - 1):
+ for expired in _all_but_last_k(previous_snaps, max(count - 1, 0)):
Cheers,
JMH.
Made a bash script as "proof of work" and now using with fzf preview or just itself. Main reason is to get more info about installing, upgrading, deleting packages - which are they.
Idea is simple: parse strings from /var/log/pacman.log after all transactions with packages are completed and updating comment in *-meta.json file. Should work as post installation hook i guess.
Is it possible to add this feature in main python code? I'm not so good in Python, so I made a bash example how it might look like.. :)
The second question is: is it possible make a comment for snapshots with trigger S
, e.g. Hourly shapshot
, Weekly shapshot
, etc?
Triyng install yabsnap with pikaur and getting this error:
ERROR: ld.so: object 'libfakeroot.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Full log is here: https://0x0.st/HJkP.txt
And the second point is - in README file in Quick Start /etc/configs/yabsnap/CONFIGNAME.conf
but right path is: /etc/yabsnap/configs/CONFIGNAME.conf
Please fix the path.
Log:
$ sudo yabsnap create-config config
[sudo] password for nebulosa:
Config directory does not exist. Use 'create-config' command to create a config.
Created: /etc/yabsnap/configs/config.conf
Please edit the file to set 'source = ' field.
Forked from #11
Currently, configs are read from /etc/configs/yabsnap, we should provide an argument to use a different location or files.
There should be a way to specify TTL, either in config or via commandline (commandline takes precedence).
Any snaps beyond TTL should be considered as expired, and cleaned up on next scheduled run.
Snaps having TTL should show it on yabsnap list
.
I was having an issue installing the latest version of yabsnap
. I would keep hitting this error here:
install: cannot stat 'artifacts/pacman/05-yabsnap-pacman-pre.hook': No such file or directory
I found that the install-to-dest.sh
script has the wrong path for the hook. It says 05 instead of 01. https://github.com/hirak99/yabsnap/blob/9dd7753165dcbe84446f89a07ff426383e2b1f18/scripts/install-to-dest.sh#L59C43-L59C43. In my case I changed the file listed in the artifacts
folder from 01 to 05.
Turns out there was some prior work to update the PKGBUILD here, #18 (comment).
I think the script should be updated to reference the correct path, but if the hook's priority has changed according to the linked comment above, maybe there should be some extra logic to remove the existing hook as well?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.