bpereto / borg-hive Goto Github PK

borgmatic is a client-driven backup solution. in some cases (ex. server farm) a central approach and central control is desirable.

the project could be extended with some scheduling functionality to trigger a backup via REST. A solution for this could be a Ansible AWX integration via REST to start some ansible-playbooks to start borgbackup.

Hi,

it is not clear (at least to me after searching the docs) if borg-hive can be used to manage existing backups!?

At the moment I use borgmatic (locally installed on my machine) to backup my stuff to my server.
I backup stuff from the server also with borgmatic to another disk of the same server (I know, not the best idea 😅 )
Now I want to run borg-hive on the server as well and have it manageing my backups.

Is that even possible or do I have to setup everything from scratch to use borg-hive?

When creating a superuser using

   # change into app container
   docker exec -it borg-hive_app_1 /bin/bash

   # create superuser
   ./manage.py createsuperuser

---

root@3bf00ebcbdd4:/app# ./manage.py createsuperuser
Benutzername (leave blank to use 'root'): borg
E-Mail-Adresse: 
Password: 
Password (again): 
Dieses Passwort ist zu kurz. Es muss mindestens 8 Zeichen enthalten.
Dieses Passwort ist zu üblich.
Dieses Passwort ist komplett numerisch. 
Bypass password validation and create user anyway? [y/N]: y
Superuser created successfully.
root@3bf00ebcbdd4:/app# 

this user is presented with German.

Should be English by default.

As of 8th of November, all my clients are getting a Permission denied (publickey). from their keys.

Any idea what could cause this?

borgbackup emits as few information as possible to the remote end (borg-hive in this case).
Discussion to this is here: borgbackup/borg#5173

Alternativ idea to log or provide more information to borg-hive is a client side hook. borgmatic provides a hook system, which could be extended for borg-hive.
https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/

this opens the possibility to store the borgmatic output in borg-hive or parse the repository information from it:

• how many backups
• timestamps
• repo size
• repo compressed size
• repo deduplicated size

A server-side way would be more flexible and would avoid specific implementations for clients.

On K8s the Location name is "localhost".

Should be configurable by borg instance?

Is it possible to backup the Hive itself? I can't find where/what I should backup to keep the settings and SSH keys I've entered, etc.

Watcher crashes on repository create:
borg init --encryption=repokey-blake2 bqs6jumh@localhost:test4

watcher_1  | Call failed (should not be -1): (-1) ERRNO=(0)
watcher_1  | Traceback (most recent call last):
watcher_1  |   File "/app/borghive/management/commands/watch_repositories.py", line 51, in handle
watcher_1  |     for event in i.event_gen(yield_nones=False):
watcher_1  |   File "/usr/local/lib/python3.8/site-packages/inotify/adapters.py", line 300, in event_gen
watcher_1  |     self._i.add_watch(full_path, self._mask)
watcher_1  |   File "/usr/local/lib/python3.8/site-packages/inotify/adapters.py", line 95, in add_watch
watcher_1  |     wd = inotify.calls.inotify_add_watch(self.__inotify_fd, path_bytes, mask)
watcher_1  |   File "/usr/local/lib/python3.8/site-packages/inotify/calls.py", line 34, in _check_nonnegative
watcher_1  |     raise InotifyError("Call failed (should not be -1): (%d)" %
watcher_1  | inotify.calls.InotifyError: Call failed (should not be -1): (-1) ERRNO=(0)

Hello,

thanks for sharing borg-hive.
It would be good, if you could add install instructions to Readme.md.

Regards,
Hendrik

Helm should generate a secret on install and django should create a superuser if secret defined.

• This ensures no weak default password is set.
• This removes the need to kubectl/docker into the container and run ./manage.py createsuperuser

Make Borgbackup Version selectable per Borg Repository.

• Build Borg Docker Image with multiple Borgbackup versions in alternate installation paths
  • /opt/borgbackup1.0
  • /opt/borgbackup1.1
  • /opt/borgbackup1.2

Is it possible to set a timezone for the interface, so it's not all in UTC showing to the users?

Hi,

My watcher keep crashing as soon as i init at repo, it caused by a permission issue :

watcher_1  | PermissionError: [Errno 13] Permission denied: '/repos/2fl9gc41/docker'

It seams that borg init the repo with this permission :

drwx------ 3 1114 1000 4.0K Apr 13 15:07 docker/

So the watcher can't read it

Add Helm Charts for k8s deployment

Hi everybody, happy holidays to all.

Here testing the project with Docker (docker-compose). I left all the variables in the .env file as they come by default including the BORG_SSH_PORT=2222.

The web dashboard opens fine at http://localhost:8000/ and I can login with the admin user and admin password declared in the .env file.

I could create a repository but when I execute borg init --encryption=repokey-blake2 dzkch3wy@localhost:my_first_repo as the UI suggest the shell ask me for the SSH password of the user dzkch3wy. Where can I found/create that password? I'm missing something in the doc?

NOTE: I do not followed the steps in https://borg-hive.readthedocs.io/en/latest/develop.html as I think this is only for a development environment.

The release mentions a major release - REST API...

But what is it, and where can we find it?

a mechanism to import or export the borg repository is desirable.

this example enables access with rsync:
https://serverfault.com/a/965929

The force command option in the ssh-keys, which is already used to enforce borg serve could also be used to enforce rsync (or a restricted version rrsync) to only allow rsync with selective read-only (export) and write-only (import)

command="/path/to/rrsync -wo /allowed/directory/",restrict,from="a.b.c.d" ecdsa-sha2-nistp521 AAAAE...

it's a pity that the rrsync script is in perl.. but should do its job
https://ftp.samba.org/pub/unpacked/rsync/support/rrsync

I suggest the following:

• the import / export mode can be activated per repository
• Import Mode enables rrsync (write-only) command in the ssh-key
• Export Mode enables rrsync (read-only) command in the ssh-key
• Only one Mode can be active (import mode, export mode, borg serve)

On repository create time, borgbackup naturally creates the repository with limited access permissions (only owner), thus borgbackup cannot read the repository.
Should be adressed.

At the moment the workaround is to chmod -R manually in the docker container / volume.

BorgWatcher keeps throwing this error:

Traceback (most recent call last):
  File "/app/borghive/management/commands/watch_repositories.py", line 55, in handle
    for event in i.event_gen(yield_nones=False):
  File "/usr/local/lib/python3.8/site-packages/inotify/adapters.py", line 300, in event_gen
    self._i.add_watch(full_path, self._mask)
  File "/usr/local/lib/python3.8/site-packages/inotify/adapters.py", line 95, in add_watch
    wd = inotify.calls.inotify_add_watch(self.__inotify_fd, path_bytes, mask)
  File "/usr/local/lib/python3.8/site-packages/inotify/calls.py", line 34, in _check_nonnegative
    raise InotifyError("Call failed (should not be -1): (%d)" %
inotify.calls.InotifyError: Call failed (should not be -1): (-1) ERRNO=(0)

According to the internet, it's because the limit on inotify... I've upped this to the max, but it keeps happening... Any idea?

Hello,

thanks for providing the docker-compose.yml.

I would consider it useful - especially for users wanting to test, rather than developers - to provide a docker-compose.yml that uses the already built container(s) from the docker hub:

• https://hub.docker.com/r/bpereto/borg-hive
• some borg container available from docker hub
  or alternatively (because most will have borg installed already)
  the borg installation of the host

Ideal would be, if one would only have to run docker-compose up after adding the /path/to/repo to the .env.

Greetings,
Hendrik

Add Notification type for GET and POST Webhooks

• Basic HTTP Request Webhook with notification payload. Questionable what should be submitted by a GET
• Webhook should be secured
  • send token (secret) to be able to verify request
• Avoid common pitfalls
  https://zapier.com/help/doc/common-problems-webhooks
• https certificates should be verified

Add REST API for managing and accessing borg-hive

• Django REST Framework
• Expose all models

Add documentation about borg-hive internals.

• Overview of architecture
• Image
• Connection overview

Hello,

I have some repositories locally, but also some that are reachable via a remote borg instance.
Will these also be visible in borg-hive?

Regards,
Hendrik

Where can I configure the email notifications SMTP server? It seems unwilling to email at the moment.