kenkundert / emborg Goto Github PK

I would like to automatically pass '--stats' to the prune function when I run 'emborg -v --narrate create' with prune_after_create enabled in the settings file. I like to see what emborg and borg are doing (though '--list' for prune might be too much info). I would guess that the easiest way might be to change the default behaviour of verbose to include stats for prune. But I don't know it that is true, how to do it myself, or if others would appreciate that default behaviour.

CREATE BACKUP WITH PORT 2222 WITH BORG
borg --verbose init --encryption=none test
sudo borg create --list --show-rc --verbose --progress --stats --comment "Backup Now." ssh://[email protected]:2222/test::Backup-{now:%Y-%m-%dT%H:%M:%S.%f} /home/test

INFO ON BACKUP USING EMBORG OR ANY COMMAND
Note:Using ssh_command = "ssh -p 2222"
emborg info

Warning: The repository at location ssh://[email protected]/test was previously located at ssh://[email protected]:2222/test
Do you want to continue? [yN] Aborting.
Repository access aborted

The Emborg Repository location won't include the port and borg will see it as a different location.

I tried:
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no

with limited success and would prefer not to if there is a better option.

It took me a while to figure out that the following error was refering to a problem with my avendesora config, not with emborg or borg itself.

$ emborg init
emborg warning: rsync: url attribute found, should probably be urls.
emborg error: Connection closed by remote host. Is borg working on the server?

Hi, I need to add a folder which has spaces in its name to the pattern of folders to backup.

In example, how can I add ~/.config/My Dir/ to my pattern which i.e. is:

patterns = '''
    R ~/Documents
    R ~/Music
    - ~/Documents/work
'''

Hi,

I can't seem to get the "passcommand" config directive to work as it does with plain Borg itself, but perhaps I am doing something incorrectly that is Emborg-specific, but wasn't sure if it was a bug?
passcommand = 'gpg -qd /home/user/.store-auth.gpg'

I've always had this same setup working with plain Borg itself, as well as with Restic and Rclone using these env vars:

BORG_PASSCOMMAND=gpg -qd /home/user/.store-auth.gpg
RCLONE_PASSWORD_COMMAND=gpg -qd /home/user/.store-auth.gpg
RESTIC_PASSWORD_COMMAND=gpg -qd /home/user/.store-auth.gpg

So it works fine with plain Borg and others, but not with Emborg (neither the "BORG_PASSCOMMAND" env var, nor the explicit Emborg "passcommand" config directive). Everything else with Emborg works fine, so long as I don't use any encryption. But with encryption, I have to use plain Borg directly.

I get this error with verbose enabled:

$ emborg -vc vault init
emborg error: Cannot determine the encryption passphrase.

The log largely says the same:

failure detected:
    Cannot determine the encryption passphrase.
emborg error: Cannot determine the encryption passphrase.
emborg: terminates with status 2 on Tuesday, 17 March 2020 at 11:32:27 AM EDT.

I get the same behavior on both Linux (Manjaro) and MacOS (Catalina/10.15), both with the same Emborg/Borg versions:

$ emborg version
emborg version: 1.15.0 (2020-03-06) [Python 3.8.1].

$ borg --version
borg 1.1.10

Would you consider adding an option for borg check --repair as detailed here?

Maybe by using emborg check --repair or even emborg repair.

Thank you!

Emborg 1.3.0

File "/home/foo/.local/lib/python3.5/site-packages/emborg/settings.py", line 112
    f'Configuration directory created: {parent!s}.',
                                                  ^
SyntaxError: invalid syntax

I think it would be helpful to have the ability to set the -v, --verbose options as default, maybe from withing the config files.

Given multiple configurations on ~/.config/emborg/, let's call them home, lamp, data, and so on, I think it would be helpful to have an option to run all found configs with a single command.

So, instead of running emborg -c home && emborg -c lamp && emborg -c data, maybe one could run emborg -a or emborg --all, or emborg or something similar.

I found out that while you have an archive mounted you get errors while trying to create new backups:

➜ emborg -c linked_path create
emborg error: borg create:
    Failed to create/acquire the lock /home/stratos/borg_backups2/serenity-stratos-linked_path/lock (timeout).

The message was confusing enough and it took me QUITE some time until I figured out that it might be due to the currently mounted archive. Once I emborg unmounted the archive the timeouts were gone.

Can this particular case be captured, and maybe a more informative message shown? Maybe even something "Do you currently have any archives mounted?".

It seems that if I try to restore files from a linked folder I get an error.

A demonstration of the failure:

# First create the folder structures
root@yoda:/# cd /tmp
root@yoda:/tmp# mkdir -p folder/subfolder
root@yoda:/tmp# touch folder/subfolder/file.txt
root@yoda:/tmp# ln -s folder linked_folder


# Show the emborg configuration files
root@yoda:/tmp# cat ~/.config/emborg/plain_path
src_dirs = '/tmp/folder/subfolder'           # absolute path to directory to be backed up
root@yoda:/tmp# cat ~/.config/emborg/linked_path                                                                                                                                                                                           
src_dirs = '/tmp/linked_folder/subfolder'           # absolute path to directory to be backed up


# Create the two borg backups and show their manifests
root@yoda:/tmp# emborg -c plain_path create
root@yoda:/tmp# emborg -c linked_path create                                                                                                                                                                                               
root@yoda:/tmp# emborg -c plain_path manifest
Archive: yoda-2021-08-19T20:40:10
tmp/folder/subfolder
tmp/folder/subfolder/file.txt
root@yoda:/tmp# emborg -c linked_path manifest                                                                                                                                                                                             
Archive: yoda-2021-08-19T20:40:27
tmp/linked_folder/subfolder
tmp/linked_folder/subfolder/file.txt


# Perform restore on the original (unlinked) folder
root@yoda:/tmp# cd folder/subfolder
root@yoda:/tmp/folder/subfolder# ls -al
total 0
drwxr-xr-x  2 root root  60 Aug 19 20:39 .
drwxrwxrwt 14 root root 300 Aug 19 20:41 ..
-rw-r--r--  1 root root   0 Aug 19 20:39 file.txt
root@yoda:/tmp/folder/subfolder# mv file.txt file.txt.deleted
root@yoda:/tmp/folder/subfolder# emborg -c plain_path restore file.txt
Archive: yoda-2021-08-19T20:40:10
root@yoda:/tmp/folder/subfolder# ls -al
total 0
drwxr-xr-x  2 root root  80 Aug 19 20:42 .
drwxrwxrwt 14 root root 300 Aug 19 20:42 ..
-rw-r--r--  1 root root   0 Aug 19 20:39 file.txt
-rw-r--r--  1 root root   0 Aug 19 20:39 file.txt.deleted


# Try to perform the same on the symlinked folder
root@yoda:/tmp/folder# cd ../..
root@yoda:/tmp# cd linked_folder/subfolder
root@yoda:/tmp/linked_folder/subfolder# ls -al
total 0
drwxr-xr-x  2 root root  80 Aug 19 20:42 .
drwxrwxrwt 14 root root 300 Aug 19 20:42 ..
-rw-r--r--  1 root root   0 Aug 19 20:39 file.txt
-rw-r--r--  1 root root   0 Aug 19 20:39 file.txt.deleted
root@yoda:/tmp/linked_folder/subfolder# rm file.txt
root@yoda:/tmp/linked_folder/subfolder# emborg -c linked_path restore file.txt
Archive: yoda-2021-08-19T20:40:27
emborg error: borg extract: Include pattern 'tmp/folder/subfolder/file.txt' never matched.

It seems like emborg is trying to match the original path (the path that the symbolic link is pointing too) instead of the actual path.

Isn't this a bug on emborg?

Thanks, btw for an otherwise cool utility! :)

Currently emborg mount accepts a --date option where you can select a given date and mount the archive(s) found for that particular timeframe.

I think it would be useful to also have an option to mount the most recent archive found in the repository without having to explicitly give a date.

A possible suggestion would be --last, --latest, or something similar.

Thank you.

If several hostnames are saved in the same repository starting their archive names with {host_name}-, then for pruning, it is necessary to add {host_name}- as a prefix with the --prefix option.

See in Borg quickstart, the prefix is said to be "very important". Using prune without prefix in emborg, I inadvertently pruned other hostname's archives.
Then, one could argue it is more efficient to create one repository per server:

• there is no risk of having failure due to simultaneous backups,
• the local cache of checksums does not need to be rebuilt every time another server makes a backup.

However this prevents deduplication across servers and when space is the priority having one single repo makes sense.

Some notes for the implementation:

1. Having prefix as a command line option is dangerous, if one forgets it with simply emborg prune, it results in the pruning of the repo without filtering.
2. --prefix is used for pruning and listing, having a general prefix option in the repo settings that applies to both will prevent listing any other archives, which can be detrimental. In this case we could offer a --ignore-prefix option that ignores the prefix provided in the setting, it could apply to list and even prune: emborg --ignore-prefix list will list all archives ignoring the prefix in settings.
3. Last, we could have a prune_prefix and a list_prefixoption, this makes it a bit more complicated but addresses the points.

I would go for 2) which sticks as much as possible to Borg naming of options and logic.
#19 is a first step for implementing 2) which should not create any issue, the next step could be to create the '--ignore-prefix' and extend the option to list as well. However this simple option addition already reaches the goal for safe pruning.

With the addition of the hooks, emborg now depends on requests also. However, this dependency is not mentioned in setup.py

It would be nice to have a command to check an Emborg configuration for gross errors that are hard to detect without watching a backup (with -v) while it is running. Detecting cases where the src_dirs are empty (could indicate an unmounted source), where exclusions can't match any path specified in the src_dirs, or where exclusions exclude every file, and so forth, would reduce the risk of mistakes leading to unusable backups or wasted storage space.

In my existing borg backup scripts, I like to see the results of a prune. To achieve this, I pass '--list' to the prune command.

Can this option be added to emborg prune also? It's available as an option to create.

Possible enhancement is to have in configuration exclude_from accepting list of filenames instead of a single name as Borg accepts several instances of exclude_from.

I need to backup a folder which is located inside an excluded folder.
More precisely, I need to backup ~/.cache/evolution/, while ~/.cache/ is excluded.

This is my current configuration

src_dirs = '''
    ~/
    ~/.cache/evolution/
    /home/shared/
'''.split()   # absolute path to directory to be backed up

excludes = '''
    ~/**/__pycache__
    ~/**/*.pyc
    ~/**/.*.swp
    ~/**/.*.swo
    ~/**/node_modules
    ~/.cache
    ~/.npm
    ~/.pnpm-store
    ~/.local/share/baloo
    ~/.local/share/Trash
    ~/.thumbnails
'''.split()

But it does not seem to work. Is it possible to do so? And how?

The default configuration files created by running emborg init are not able to interpret source directories or exclude masks that include spaces or single quotes. The Python split() method does not have provision for escaping spaces or other characters. As a result, including a path with a space in an Emborg configuration file will result in Emborg attempting to parse the path as two, incomplete, paths, with undesirable results.

If the configuration files are rewritten to use Python list syntax, rather than split(), Emborg processes paths with spaces correctly. For example, this configuration does work without any changes to Emborg proper:

src_dirs = ['/mnt/drive to back up', '/mnt/other drive to back up' ]
excludes = [
'*/desktop.ini',
'*/Thumbs.db',
'*/stuff to exclude/*'
]

Note however, that if there are any syntax errors in the exclusion list (such as a missed trailing comma), Emborg will execute a backup but ignore the entire exclusion list. Presumably this is also true if using the standard configuration files. I feel syntax errors in the configuration files should be treated as fatal errors.

There is a new --latest option for mounting the latest archive, however by default other commands like extract and manifest apply to the latest archive while mount by default mounts all archives.
Therefore wouldn't it make more sense that mount by default mounts the latest archive and that mount --all mounts all archives?
Just a question as I may miss some logic behind.

Related to #21 (comment)

I did an 'emborg manifest' on a raspberry pi running raspbian 10.3 (a derivative of debian 10.3) and saw the following traceback. The files were not listed by the manifest call.

  File "/usr/local/bin/emborg", line 10, in <module>  sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/emborg/main.py", line 96, in main cmd_name, args, settings, emborg_opts
  File "/usr/local/lib/python3.7/dist-packages/emborg/command.py", line 162, in execute  exit_status = cls.run(name, args if args else [], settings, options)
  File "/usr/local/lib/python3.7/dist-packages/emborg/command.py", line 1127, in run sep="\n",
  File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 1452, in __call__   INFORMER._report(args, kwargs, self)
  File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 1936, in _report multiline, continuing, options
  File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 2028, in _show_msg
 print(': '.join(cull([header, culprit, message])), **options)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0151' in position 82: ordinal not in range(256)
root@raspberrypi:~# client_loop: send disconnect: Broken pipe

I'm trying to upgrade the Alpine package to the latest version, and add in the ability to run the tests during the package build.

I'm finding that a number of the tests are failing due to the way the Alpine CI process works (doing the build inside a Docker container, hence meaning the borg mount tests will fail as FUSE isn't available).

You may be able to see the output of a build here: https://gitlab.alpinelinux.org/adhawkins/aports/-/jobs/473511

Is there any way to skip selected tests with the test system you're using?

I realize this is probably more of a forum-question kinda thing, but I couldn't find one, so here goes nothing:

I created a repo using emborg and uploaded some test data to it (rsync.net account). I was warned to extract my key, so I tried doing that:

pi@raspberrypi:~/.config/emborg $ emborg borg key export [email protected]:borg_rpi borg.ke
Local Exception
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4455, in main
    exit_code = archiver.run(args)
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4387, in run
    return set_ec(func(args))
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 154, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 358, in do_key_export
    manager.export(args.path)
  File "/usr/lib/python3/dist-packages/borg/crypto/keymanager.py", line 85, in export
    self.store_keyfile(path)
  File "/usr/lib/python3/dist-packages/borg/crypto/keymanager.py", line 81, in store_keyfile
    with open(target, 'w') as fd:
PermissionError: [Errno 13] Permission denied: 'borg.ke'

Platform: Linux raspberrypi 5.4.79-v7l+ #1373 SMP Mon Nov 23 13:27:40 GMT 2020 armv7l
Linux: debian 10.9
Borg: 1.1.9  Python: CPython 3.7.3
PID: 3949  CWD: /
sys.argv: ['/usr/bin/borg', '--remote-path', 'borg1', 'key', 'export', '[email protected]:borg_rp
i', 'borg.ke']
SSH_ORIGINAL_COMMAND: None

emborg error: unexpected exit status (2)

It seems the process CWDs to ´/´ and then has a hard time creating a file there. Does that mean that I have to use sudo when extracting the key?

emborg info shows me where the keyfile is saved: /home/pi/.config/borg/keys/..., can't I just copy the keyfile from there?

Thanks for your help!

I have the following settings:

root@raspberrypi4:/home/pi/.config/emborg# cat settings 
# These settings are common to all configurations

# configurations
configurations = 'docker'
default_configuration = 'docker'

# encryption
encryption = 'keyfile'            # borg encryption method
    # Common choices are 'repokey' and 'keyfile'.
    # With 'repokey' the encryption key is copied into repository, use this
    # only if the remote repository is owned by you and is secure.
    # With 'keyfile' the encryption key is only stored locally. Be sure to
    # export it and save a copy in a safe place, otherwise you may not be
    # able to access your backups if you lose your disk.
# specify either passphrase or avendesora_account
passphrase = 'xxx'              # passphrase for encryption key
avendesora_account = '<<account-name>>'  # avendesora account holding passphrase

# basic settings
# specify notify if batch and notifier if interactive
notify = '[email protected]'        # who to notify when things go wrong
notifier = 'notify-send -u normal {prog_name} "{msg}"'
                                         # interactive notifier program
remote_ratelimit = 2000                  # bandwidth limit in kbps
prune_after_create = True                # automatically run prune after a backup
check_after_create = 'latest'            # automatically run check after a backup

# repository settings
repository = 'xxx:/home/xxx/borg_backups/{host_name}-{user_name}-{config_name}'
prefix = '{host_name}-'
    # These may contain {<name>} where <name> is any of host_name, user_name,
    # prog_name config_name, or any of the user specified settings.
    # Double up the braces to specify parameters that should be interpreted
    # directly by borg, such as {{now}}.
compression = 'lz4'

run_before_backup = 'service docker stop'
run_after_backup = 'service docker start'

healthchecks_uuid = 'xxx-xxx-xxx-xxx-xxx'

# filter settings
exclude_if_present = '.nobackup'
one_file_system = True
exclude_caches = True

# prune settings
keep_within = '1d'                       # keep all archives created in interval
keep_hourly = 48                         # number of hourly archives to keep
keep_daily = 14                          # number of daily archives to keep
keep_weekly = 8                          # number of weekly archives to keep
keep_monthly = 12                        # number of monthly archives to keep
keep_yearly = 1                          # number of yearly archives to keep

When I run emborg create, my docker service is not stopped while the backup takes place (I checked from another terminal). I assume the run_after_backup is not run neither.

Also, when I emborg settings I see:

root@raspberrypi4:/home/pi/.config/emborg# emborg settings
                   archive: '{host_name}-{{now}}'
                            'raspberrypi4-{now}'
        avendesora_account: '<<account-name>>'
        check_after_create: 'latest'
               compression: 'lz4'
                config_dir: '/root/.config/emborg'
               config_name: 'docker'
            configurations: 'docker'
     default_configuration: 'docker'
                encryption: 'keyfile'
            exclude_caches: True
        exclude_if_present: '.nobackup'
                  excludes: """
                                **/*~
                                **/__pycache__
                                **/*.pyc
                                **/.*.swp
                                **/.*.swo
                            """
                  home_dir: '/root'
                keep_daily: 14
               keep_hourly: 48
              keep_monthly: 12
               keep_weekly: 8
               keep_within: '1d'
               keep_yearly: 1
                   log_dir: '/root/.local/share/emborg'
                  notifier: 'notify-send -u normal {prog_name} "{msg}"'
                    notify: '[email protected]'
           one_file_system: True
                passphrase: '<set>'
                    prefix: '{host_name}-'
                            'raspberrypi4-'
        prune_after_create: True
          remote_ratelimit: 2000
                repository: 'xxx:/home/xxx/borg_backups/{host_name}-{user_name}-{config_name}'                                                                                                       
                            'xxx:/home/xxx/borg_backups/raspberrypi4-root-docker'
                  src_dirs: """
                                /var/lib/docker/volumes/xxx1
                                /var/lib/docker/volumes/xxx2
                                /var/lib/docker/volumes/xxx3
                            """

Same issue with the healthchecks_uuid config. There is no ping taking place!

emborg is run as the root user (so I can read the root owned docker volumes)

What am I doing wrong and why does emborg does not "see" my settings?

I'm not sure how many active users emborg has, but have you considered creating an IRC channel for people to discuss it and get help? I think it'd be a good idea. Librera seems to be the IRC network of choice these days.

There's more info on registering a channel here:

https://libera.chat/chanreg/

I imagine it's a relatively straightforward process.

# emborg help

Traceback (most recent call last):
  File "/usr/local/bin/emborg", line 7, in <module>
    from emborg.main import main
  File "/usr/local/lib/python3.5/dist-packages/emborg/__init__.py", line 4, in <module>
    from .settings import Settings as Emborg
  File "/usr/local/lib/python3.5/dist-packages/emborg/settings.py", line 112
    f'Configuration directory created: {parent!s}.',
                                                  ^
SyntaxError: invalid syntax

emborg was installed as pip3 install emborg, not pip3 install --user emborg because doing so, as root for install and as root to run, yield nothing.

Could this be because borg was installed as a single binary?

/usr/local/bin/borg -> /folder/where/borg-binary-is/borg-linux64

I would like to pass the '--progress' option to Borg when running emborg create. This will allow me to monitor the progress of an Enborg backup (which I usually run interactively.) It would also be cool to have a show_progress setting in the config file allowing the user to configure Emborg to always show progress. Basically, I would like the same modification that was done in #28, but for the '--progress' options.

You can find out more about Borg's progress option here. (I've also quoted the text below.)

The --progress option shows (from left to right) Original, Compressed and Deduplicated (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path.

when doing emborg prune, the comments returned for prune are wrong

I attempted to backup my computer this morning with the emborg create command, but it failed.
Judging from the error messages (see below), it seems to be an issue with borg on rsync.net's servers which they updated this morning. I'm not sure if there is much you can do from emborg to fix it, but I thought I should let others know. (Or you can correct me if I'm wrong and there is an issue with Emborg)

Here (log.txt) is the output of the create command with the verbose and narrate options. I have removed some things indicated by <<< >>>. The TypeError complaining about the number of arguments has always occurred in the past during successful runs.

Before actually reading the docs for the diff command, I was thinking/hoping that it would do a diff of the contents of the current dir against the latest backup, but alas it does not... :(

My use case, is that I want to easily see if the contents of the current dir have been changed since my last backup. I never thought that I will need the diffs between two arbitrary previous backups (although useful for some cases, I guess).

My suggestion would be that if you pass NO arguments to diff then you compare the last backup with the current folder (and subfolders, I would guess)

If you pass a single parameter (of any previous backup) you can diff that backup with the current folder.

Passing two params is what you currently have.

As a workaround, for now I mount the latest backup in a random folder and I manually diff the folders (having to laboriously copy the full path to the folders I need to compare). Doable, but I believe (as I originally thought) this functionality should be native to emborg.

BTW, while I was doing the above mounting exercise I came across [another issue (#48) (I believe)...

I am a great fan of emborg, I have created and maintain the Arch Linux package.
I have noticed the addition of an external dependency to quantiphy in the latest version and have therefore created a new package for it.
I just wanted to highlight this makes the installation of emborg dependent to one more "unofficial" package. Not an issue at all, just for your information, dependencies to non-standard python packages is not neutral.
Thanks again for what you are offering with emborg

this issue can be closed as it does not require specific action, just fyi.

Is it possible to make optional email notifications upon successful completion of a backup?

Borgmatic has a really nice feature where it will do a POST to healthchecks.io upon job completion. The POST body contains the stats of the job, not just the exit value. It would be nice if Emborg had this ability as well.

I often use the borg command on the repo using the @repo variable, however how can I run a borg command on a particular archive of the repo? I have tried several combinations like @repo::Monday without success.

Healthchecks.io can be self-hosted. It would be useful if emborg could allow the ping URL for healthchecks.io to be specified, allowing a self-hosted solution to be used.

Trying to package up 1.29 for alpine, and running in to the following error when running the tests:

____________________ ERROR collecting tests/test_emborg.py _____________________
ImportError while importing test module '/home/builder/aports/community/emborg/src/emborg-1.29/tests/test_emborg.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_emborg.py:15: in <module>
    from shlib import Run, cd, cp, cwd, ln, lsf, mkdir, rm, set_prefs, touch
/usr/lib/python3.10/site-packages/shlib/__init__.py:5: in <module>
    from .shlib import (
E   ImportError: cannot import name 'brace_expand' from 'shlib.shlib' (/usr/lib/python3.10/site-packages/shlib/shlib.py)
=========================== short test summary info ============================

I can successfully build and test 1.28, so am pretty sure that my build is Ok. There don't seem to be any new dependencies in emborg, and shlib doesn't seem to have changed since the release that's being installed in alpine.

Any suggestions?

Firstly, thanks for releasing Emborg!

The documentation at https://github.com/KenKundert/emborg/blob/c9cc1ace8758ad334bb929ae92ed98ee62988c78/doc/index.rst states the following:

repository: This is the location where all of your files are backed up to. It may be on a local file system or it may be remote, in which case it is accessed using ssh.

Which implies SSH is not used for repositories on the local filesystem. However, if I create a repository of the form: repository = 'mylocalhostname:/backup:/{host_name}-{user_name}-{config_name}' it still tries to connect over SSH:

/backup $ emborg init
The authenticity of host 'mylocalhostname (::1)' can't be established.
ECDSA key fingerprint is SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXX.
Are you sure you want to continue connecting (yes/no)?

What's the correct way to tell Emborg to use a local repository, so that it doesn't use SSH? Is there something else that needs to be put in the first field (n.b. "localhost" yields the same problem)?

Depending on making a successful SSH connection via the loopback interface adds a superfluous dependency on SSH, and will break if a user does anything with SSH locally (changes the host keys, disables it, etc). If there's a way to avoid this, that would be great!

There is no way to use the {health} format specifier when using the 'manifest' command in emborg.

It would be useful if this could be added.

Hello,

I've been trying out emborg as a frontend for borg.
But it seems like most examples listed in the docs are not working (making it difficult to see archives, restore etc)
I ran the following emborg commands:

$ emborg -c test init # created the remote borg repo
$ emborg -c test create

Running Borg create command ...
Checking archive ...
emborg error: no archives are available.
     This error occurred while checking the archives.
    No error was reported while creating the archive.
   
# emborg can't find the archive created?
$ emborg -c test list


# but borg can?
$ emborg -c test borg list @repo
 serv-2021-03-07T17:23:162021-03-07T17:23:16 Sun, 2021-03-07 17:23:17 [9f42bb5ff2affc23e7ac21dd8550cd2762043e3d27bcd1e113317f05214b247f]
 serv-2021-03-07T17:23:522021-03-07T17:23:52 Sun, 2021-03-07 17:23:53 [8bbab3b27eab798d458b4b9aef9aec6e2a8baf69a375f0e107ad77ae91083a76]
 serv-2021-03-07T17:28:552021-03-07T17:28:55 Sun, 2021-03-07 17:28:56 [560e3a7c630f51b0298543728b3f246899b12d6af914b13e69e4705c538fd255]
 serv-2021-03-07T17:49:232021-03-07T17:49:23 Sun, 2021-03-07 17:49:24 [5810e276481b1020189a0c7218543f203a43d55fad4063f1070c0862bcc6acee]

$ mv test_dir test_dir_2

# because emborg can't find the archive I can't restore it :/
$ emborg -c test restore test_dir
emborg error: no archives are available.

$ emborg version
emborg version: 1.20.0 (2021-02-13) [Python 3.9.2].

It seems that emborg can't find the archives?
Even though emborg created them and that borg can find the archives.

Extra info

The test configuration is stored in ~/.config/emborg/test

# Settings for a test configuration
# use of absolute paths is recommended
src_dirs = '/home/zeus/vpn'           # paths to directories to be backed up

The general settings file is stored in ~/.config/emborg/settings
And looks as followed (omitting sensitive data)

# These settings are common to all configurations

# configurations
configurations = 'home root test'
default_configuration = 'home'

# encryption
encryption = 'repokey'            # borg encryption method
    # Common choices are 'repokey' and 'keyfile'.
    # With 'repokey' the encryption key is copied into repository, use this
    # only if the remote repository is owned by you and is secure.
    # With 'keyfile' the encryption key is only stored locally. Be sure to
    # export it and save a copy in a safe place, otherwise you may not be
    # able to access your backups if you lose your disk.
# specify either passphrase or avendesora_account
passphrase = '<my_passphrase>'              # passphrase for encryption key

# basic settings
# specify notify if batch and notifier if interactive
notify = '<my_mail>'        # who to notify when things go wrong
notifier = 'notify-send -u normal {prog_name} "{msg}"'
                                         # interactive notifier program
remote_ratelimit = 2000                  # bandwidth limit in kbps
prune_after_create = True                # automatically run prune after a backup
check_after_create = 'latest'            # automatically run check after a backup
show_progress = True

# repository settings
repository = 'backup@<server>:/home/backup/Sync/borg/{host_name}-{user_name}-{config_name}'
prefix = '{host_name}-{{now}}'
compression = 'zstd'

# filter settings
exclude_if_present = '.nobackup'

Any help would be greatly appreciated.

PS: Running any option with the --verbose flag simply prints Running borg <command> command ... and nothing more

It would be nice to use also the --exclude-from file option of Borg, this avoids repeating the same exclude paths in several setting files and should be relatively straightforward to implement.
I can help if needed.

Hello! A small question on restoring from backups. I have several computers from which I backup to my server. In the emborg settings on computers, I specified encryption: 'repokey' and the passphrase. Suppose that one of the computers is completely out of order, i.e. the emborg settings were lost, but I remember the passphrase. How can I access computer backups on the server where the repositories are located?

Hi,

is there already a solution to backup to multiple repositories? E.g. BorgBase and another internal repository?

Kind regards,
Andreas

Typo in the title of the release. Thought I'd point it out!

I currently run my emborg jobs individually from a script, that does a 'create' and 'prune' for each job, and then emails the results of that individual job before moving on to the next one.

Sometimes I want to exclude some jobs from the backup run. Currently the only way I can do that is to move the file out of the config directory.

It would be helpful if each individual job's settings file had an entry along the lines of 'disable: false / true', which would just (say) print out a message saying 'This job is disabled' if certain operations (say create, check and prune) were run on it.

Would that be useful generally do you think?

If borg completes with a warning (say a file disappeared during the backup, so could not be processed) it appears that the healthcheck 'finished' ping isn't sent.

Borg's return codes are documented here:

https://borgbackup.readthedocs.io/en/stable/usage/general.html#return-codes

It looks like the 'success' ping is only sent if borg returns zero:

Maybe using cookiecutter

Hello!
The emborg-overdue does not send letters after completion. The logs do not even show an attempt to send a letter. At the same time, an ordinary emborg always sends letters successfully.

Config file:

default_maintainer = 'backup@localhost'
dumper = 'backup@localhost'
default_max_age = 25 # hours
root = '/mnt/backup_data/borg'
repositories = [    
    dict(host='fs (/)', path='fs/root'),
]

Messages for backup@localhost are forwarded to Gmail.

Emborg-overdue output:

HOST: fs (/)
    sentinel file: /mnt/backup_data/borg/fs/root/index.43
    last modified: 2019-08-07T09:41:23.706817+00:00
    since last change: 120.9 hours
    maximum age: 25 hours
    overdue: True

~/.config/emborg is used to save log and last backup information, this should rather go in ~/.local/share/emborg.

#20 implements this suggestion.