daknob / eldim Goto Github PK

This bug tracks the progress of adding validation checks to ensure eldim has been configured with a bucket that exists, and that is has appropriate permissions on this bucket (such as creation of files, etc.).

It is okay for now to only have this check on startup, and handle errors due to changes afterwards less gracefully.

Move eldim and client config data structures into a separate package, add Validate() as a method to the struct s directly, and use this in main package.

This will make it reusable, cleaner, and easier to deal with.

This issue tracks the progress of adding S3 support to eldim so it can store files there. The S3 protocol is the most common protocol for object storage today, and will increase eldim's capabilities by a lot.

The configuration MUST support multiple backends, and SHOULD show an error if the S3 bucket permissions allow for public reads from everyone, to prevent data leaks.

This issue tracks the progress of adding support for Google Cloud Storage backends to eldim. These buckets cannot have an expiration policy configured by eldim, but instead will use the settings from the Google Cloud Console for storage class permissions. Policies can be configured there.

The initial design of eldim included a separate list for IPv4 and for IPv6 addresses per client. This is not really needed anywhere, and instead adds complexity to the code that isn't necessary.

This bug tracks the removal of this, and the replacement by a single list, that contains both IPv6 and IPv4 addresses.

NOTE: This requires changing the configuration file format, so appropriate warnings must be added

Currently eldim uses the TripleSec encryption algorithm for stored files, which is not necessarily designed for file encryption. It requires the entire content to be in-memory for encryption to happen, and it is suspected of crashing eldim once in the past, over many years of the tool running.

There is a new file encryption standard, age, which looks very promising, and most importantly it can support asymmetric encryption, which will not require eldim servers to have the decryption keys for the data passing through eldim.

The age tool has a Go library that can handle encryption, given one or more public keys. The key files should be passed as a list of filenames via the configuration file, and eldim must encrypt the content with the given keys, so any of the keys can decrypt the end file.

The command line tool of eldim should be modified to support versions (v1 & v2), and eldim should somehow signal the version, ideally by appending a specific extension to all uploaded files, such as .eldim2. The command line tool should be able to decrypt the file based on the input file name, but must also accept explicit version to decrypt data as.

Currently eldim relies on the backend, be it OpenStack Swift or other, to accurately timestamp all uploaded files. However, for more visibility and for even more solid proofs that each file was uploaded at a particular time, eldim can make use of RFC3161 time servers.

These are publicly operated and trusted servers that sign messages given a particular hash of some content, adding a timestamp as well, to prove that this file existed at least as back as the date of the timestamp added by this server.

The configuration file should support multiple Timestamp Servers, each having a name and an address to contact. After obtaining the proper signatures, eldim should store all files obtained in the backends, using the $filename.$tssname.tsr file name.

A free timestamp server can be found here, operated by Free TSA.

Currently eldim supports only OpenStack Swift backends. It would be nice to have support for Backblaze B2 storage, so eldim users can store their files there.

The configuration should be similar to the existing OpenStack Swift backend, and support for multiple backends should exist.

Through the configuration file, the users should be able to pick whether they want the Europe or the US region, or potentially any future regions that may come up.

Optionally, eldim could check if the bucket is private and not public before uploading any files (on script startup), and log accordingly, to prevent data leaks.

Example File clients.yml

name: "mail.example.com"
ipv4:
- "192.0.2.10"

returns FATA[0000] Invalid configuration: Failed to validate Clients File: IP "192.0.2.10" in host "mail.example.com" (1) is not an IPv4 Address

Example File clients.yml

name: "mail.example.com"
ipv4:
- "192.0.2.[1-9]"

works.

It seems that parsing of the file clients.yml does not recognize ips that are above xxx.xxx.xxx.9