brxken128 / dexios Goto Github PK

Describe the bug
In src/erase.rs:32, the secure_erase function does not write random bytes for the remainder of the given input if the size of input is not a multiple of 512 bytes.
For example, if we have a file of size 20648, we iterate (data.len() / 512)*512 = 40 times. This means we write 40*512 = 20480, leaving 20648 - 20480 = 168 bytes at the end of input unmodified on disk.

To Reproduce
dexios erase <input> for file not a multiple of 512 bytes.

Expected behavior
Modify behavior of inner loop at src/erase.rs:31 to write the residual bytes after executing the for i in 0..data.len() / 512 { ... } loop.

Screenshots
N/A

Desktop (please complete the following information):
N/A

Additional context
If I've made a mistake in my assessment of the issue, please correct me.

I want to encrypt and decrypt git repo and as result to have one encryption file
How would I do it?

I think it's okay in the grand scheme of things to allow Cloneing of Protected values. Clones need to at least have some thought behind them, which should still prevent misuse, but it also allows us to re-use protected values (only when strictly required).

I ran into an issue regarding this and V5 headers - each keyslot gets a unique salt, which requires re-hashing the raw_key (user-provided key) multiple times. We can either re-use the salt throughout all of the keys (which would require a re-work) or we can go with unique salts.

I'm going to leave V5 with unique salts, although I wouldn't be opposed to changing it in the future. I understand why salts are so important, and I know that the chances of an attacker making rainbow tables and such for Balloon and argon2id algorithms are slim, but I'm still not convinced.

Update decrypt's about section within cli.rs to say "Decrypt a file" instead of "Decrypt a previously encrypted file".

The latter looks terribly out of place.

I have mentioned this in the wiki, but file metadata is not preserved once it has been archived.

I believe there is a way to do this and I will (try to) implement it shortly, just thought I'd provide a heads-up.

This will be used primarily for updating a file's header version, or just changing the nonce/salt. I do not plan to add support for modifying the AEAD used - users are free to decrypt and re-encrypt to do this, if they so wish.

Use CryptoParams arguments (and use the same values within the clap definition of the subcommand)

This will be done via a combined encrypting/decrypting function.

Data will be re-encrypted with the AEAD that the header specifies, which means multiple files should be able to be specified and updated with one command.

The salt and nonces will be changed once re-encrypted.

For streaming, one buffer will be populated with encrypted data, and it will be decrypted to another buffer within memory. It will then be encrypted, and written to the disk. This means no plaintext data will hit the disk, and it will all be kept within memory.

For memory mode, use the same principle.

Ensure that no plain, decrypted data touches the disk (and that all plaintext data is appropriately zeroed out of memory)

I feel as though dexios update file1.enc file2.enc is user-friendly enough. Either the user can specify a keyfile/environment key (provided the all of files use that same key), or they should be asked for their key each time (this should also reiterate the file name to prevent any confusion).

Describe the bug
Using Dexios built from the standalone-hash branch will produce two different hashes, based on whether you're using stream or memory hashing.

To Reproduce
Command to reproduce this behaviour:
dexios hash test.txt
dexios hash -m test.txt

Expected behavior
Hashes should be equal (I believe - maybe this is intentional from the BLAKE3 team with regards to hasher.update() and blake3::hash)

Desktop (please complete the following information):

• OS: Void Linux

Maybe this is intentional - if so, I'd advise users to just let Dexios handle which modes to use. hash-standalone implements file-size checking, just as encrypt.rs and decrypt.rs do.

Change erase block size from 512 to BLOCK_SIZE

This will make the codebase more uniform as it'll be a standard block size throughout, and it will probably increase erase mode's performance regarding larger files.

Larger block sizes increase the speed of dd, and they increased the speed of streaming modes, so I feel as though it will be a similar case here.

I will test to confirm before merging, of course.

I have been thinking about the structure of Dexios, and I feel as though it could do with being refactored. A lot of functions are monolithic, e.g. decrypt::stream_mode, and the respective decrypt::crypto::decrypt_bytes_stream_mode.

A lot of these functions could, and should, be more generic - and they handle too much IMO.

Some of these function names aren't the greatest either, so I plan a full restructuring and refactoring of the codebase.

My planned tree is as follows:

src
├── subcommands           // handles user-facing interactiveness
│   ├── mod.rs
│   ├── encrypt.rs
│   ├── decrypt.rs
│   ├── erase.rs
│   ├── header.rs         // handles header functions (e.g. strip, dump)
│   ├── key.rs            // handles retrieval of the key
│   ├── list.rs
│   ├── update.rs
│   └── prompt.rs
├── crypto                // handles the cryptographic functions
│   ├── mod.rs
│   ├── encrypt.rs
│   ├── decrypt.rs
│   ├── update.rs
│   ├── hashing.rs        // BLAKE3 and argon2id hashing
│   ├── streams.rs
│   └── primitives.rs     // this contains crypto enums and impls, and BLOCK_SIZE etc
├── global
│   ├── mod.rs
│   ├── secret.rs
│   ├── header.rs         // handles header serialization, etc
│   ├── states.rs         // for HashMode, etc
│   └── structs.rs
├── main.rs
├── cli.rs
├── params.rs             // this handles getting params from ArgMatches
└── README.md

I also plan to use our enum-based approach to our advantage, with impls where appropriate (e.g. getting the data from a KeyFile - we don't need a dedicated function for this).

I may even change our input file types to an enum, for the sake of uniformity.

I feel as though this will help the codebase out a lot, and make things more maintainable and readable. This will push back the estimated release of v8.5.0, but hopefully not by too much. It's going to be a large task though.

Remove the need for pack/unpack, and auto-detect packing when a directory is specified as the input while encrypting.

All pack-related arguments should note that they're "for use with a directory only", and have a check for this. If any are provided but the input is a file, show an error.

This ties in with V4 Headers, and the 2-byte identifier.

Update docs to mention Android is supported.

It can be installed with AR=llvm-ar cargo install dexios

Ensure to update the README and "supported operating systems" sections.

It was tested on an Android 12 device (arm64) with v0.118.0 of Termux.

Describe the bug
Decrypting a file using Dexios built from the chacha branch does not work.

The nonce and salt are both saved and read correctly (confirmed with a hex editor).

This issue happens with both the default XChaCha20-Poly1305 and the -g for AES-256-GCM switch.

This is the only issue I have found so far with the implementation (aside from implementing it idiomatically, but that's sorted now).

To Reproduce
Command to reproduce this behaviour:
./dexios encrypt -yk keyfile test.raw test.enc
and then
./dexios decrypt -yk keyfile test.enc test.raw

Expected behavior
It should decrypt the file, instead it returns an error.

Desktop:

• OS: Void Linux (GNU's libc)

Change (version_info, algorithm_info, mode_info) to a struct named HeaderPrefix

This will clean the code up rather significantly

I'm slowly working my way through things, and refactoring code where possible. I'm also thinking about the best way to isolate each part of the code, and decide how it should be in regards to being a library crate.

I have a few good ideas on how to do this. One being to expose init_x_stream() as part of the library, and then provide an example on docs.rs about how to create a working stream that will function the same as Dexios currently does.

This will allow for third party clients, similarly to what Antidote1911 has done with his tool - DexiosGUI (great work btw!)

All of the global header functions will be exposed, such as creating AAD.

I plan to use impl where possible to make things a little nicer - one idea I had in mind was to impl serialize/deserialize for the Header struct, instead of having it as a separate function. This should clean the code up nicely.

I think an impl for Header to allow a user to read could also be helpful (accepting anything that has the Read attribute, for files/cursors/etc), as it would populate the information for the user and just make things easier all around.

Maybe i missed something or what ,but....:

c:\Users\karbantartokac\Desktop\z>dex encrypt --auto 1.dat 1.enc
-960996e0m Your generated passphrase is: September
←[36mℹ←[0m Using XChaCha20-Poly1305 for encryption
←[36mℹ←[0m Encrypting 1.dat (this may take a while)
←[32m✔←[0m Successfully hashed your key [took 2.96s]
←[32m✔←[0m Encryption successful! File saved as 1.enc [took 0.60s]

c:\Users\karbantartokac\Desktop\z>dex decrypt 1.enc 1.kii
Password: September
←[36mℹ←[0m Using XChaCha20-Poly1305 for decryption
←[36mℹ←[0m Decrypting 1.enc (this may take a while)
←[32m✔←[0m Successfully hashed your key [took 2.85s]
Error: Unable to decrypt your master key (maybe you supplied the wrong key?)

Am i do something wrong ?!

This was mentioned in #100.

I think files such as cli.rs and main.rs would look exponentially nicer, and they would be easier to manage if we used the clap derive API.

We re-use a lot of arguments and it would just make the process a lot simpler, in theory.

Serialize is spelt as serialise, same with deserialise.

I feel as though this should be changed for uniformity throughout the codebase, and with other rust projects - most of them use serialize.

This is a rather trivial change, and it will be done for the release of v8.0.1.

in pack.rs add large file option to enable large files.
app crashes when it encounters large files.
let options = FileOptions::default() .compression_method(zip::CompressionMethod::Bzip2) .compression_level(Some(6)) // this is the default anyway .large_file(true) // enables support of files >4Gb at a small size penalty .unix_permissions(0o755);

Describe the bug
I want to pack my directory and Dexios doesn't erase the temporary file on failure.

To Reproduce
Preface: You have data directory with some files.

Option 1:

• Run dexios pack -py data data.enc
• Dexios will ask you for your user password, but press Ctrl+C (or Cmd-C on Mac)

Option 2:

• Run dexios pack -pyk not_exist.txt data data.enc
• Dexios cannot find keyfile and will show error.

Expected behavior
Dexios doesn't pack my folder and doesn't create the temporary file (or erase it after fail).

Actual behavior
Dexios doesn't pack my folder but creates the temporary file data.enc.xxxxxx

Screenshots

pc:~/finance$ ll | grep enc
-rw-r--r--  1 user user 552549 business.enc
-rw-r--r--  1 user user   8125 family.enc
pc:~/finance$ dexios -py business business.enc
ℹ Creating and compressing files into business.enc.HbSr3HzN
ℹ Traversing business
✔ Compressed 3 files into business.enc.HbSr3HzN! [took 0.00s]
Password:
Error: Unable to read password

Caused by:
    Error reading password from terminal
pc:~/finance$ ll | grep enc
-rw-r--r--  1 user user 552549 business.enc
-rw-r--r--  1 user user 552405 business.enc.HbSr3HzN  <<<<<
-rw-r--r--  1 user user   8125 family.enc

Desktop (please complete the following information):

• OS: Ubuntu
• Version: 20.04.4

Instead of allocating decrypted_bytes and encrypted_bytes each time, use a mut buffer (created outside of the loop, next to the read buffer)

Also rename buffer to read_buffer, that way there's minimal confusion regarding what's what.

This will allow us to zeroize the buffer at the end, ensuring plaintext data is gone from memory. Do this for both encrypting and decrypting stream modes.

For longer tasks, especially on slower machines, it would be ideal to have a progress indicator/progress bar to let the user know that things are still happening.

I think the indicatif crate will work just fine, and it should probably be feature-gated in the core library behind a visual feature. This will prevent pulling in unnecessary dependencies if the core library is used elsewhere and doesn't require this functionality.

This was mentioned in #94.

Hi,
would you consider adding support for reading data from stdin and write the encrypted stream to stdout (and vice versa). Would come in handy in pipeline jobs.

Greetings

I personally do not plan to add support for encrypting/decrypting based on a glob pattern (at this moment in time anyway). I've tried before and could not get a user-friendly implementation going.

What I do think could be valuable is encrypting a singular directory. My plan was to encrypt each file in the directory as normal (so they each have their own salt/nonce), and then to compress them into a tar or zip file. I'm leaning more towards tar.

Then, that compressed file will be encrypted once again, this time with ideally a different key provided by the user, or the same key could work but only with a different salt.

If the keys were different, a brute force attack would take considerably longer. With the same key and different salts (this is the default implementation in Dexios anyway), an attacker just has to find one key and all of the files will be decrypted. I guess functionality should be added for both use cases.

As far as I'm aware, there are no cryptographic downfalls to encrypting twice with different nonces in this case. That, and the compression imposed by taring the files should be more than enough, although I'd definitely inspect output files before releasing this feature.

I would suggest file-name encryption also, but if the whole tar file is encrypted anyway, I don't see the point.

You may track the progress of this on the directory-encrypt branch - I plan to work heavily on this feature and I don't see it taking more than 2/3 days - most of the groundwork has been laid out.

Appropriate GA tests will be added also.

Return a CryptoTimings struct from memory/stream functions, with f32 durations for specific processes.

This will have a greater impact once Dexios (eventually) splits into individual crates.

This above is not high on my list. The main reason for doing this would be to separate user-facing functions from the cryptographic ones, and it would allow for a GUI version of Dexios to be built in the future.

This will then allow us to serialize/deserialize argon2id parameters - meaning header versions will not have to increment as much.

This would involve adding argon2id's Params within the Header struct.

This would allow us to also add a switch for hardened parameters - if the user wants that bit of extra security (at the cost of performance)

I think 8 keyslots (the same as LUKS) is a suitable number. It provides a good balance between extreme header size, and flexibility for the user.

Each key = 48 bytes (as of v8.7.0), but we could easily add support for balloon AND argon with a byte identifier. I understand that some people will prefer argon2id over blake3-balloon, so adding the option won't be a bad idea.

Each key could now be prefixed with DF, and then an identifier (A3 or B4 for argon-param v3 and balloon-param v4 respectively)

e.g. [DF, A2] would signify it's a key hashed with argon2id, param version 2.

[DF, 00] should be used to specify if the keyslot is disabled or not.

This will need to be done once Dexios has split.

Documentation needs to be written regarding every function, struct, implementation and just everything within the crate. Go into as much detail as possible.

(I quite enjoy writing documentation, so this won't be an issue)

We can ensure that the input header is a Dexios header by reading the first 6 bytes, and deserialise them. If they are valid, we won't get any errors.

This isn't the most thorough check in the world, but it could easily catch some mix-ups.

This will be done later, I just felt like it deserved it's own issue.

File is created in filesystem while using the -b benchmark flag
When using an input file larger than memory mode limit a 0kb is created in the filesystem.

To reproduce
Command to reproduce this behaviour:
In empty folder
create testfile then
dd bs=10M seek=1 of=testdata count=0
./dexios -eb testdata out

Should be a very simple fix

I think it would be very advantageous to support encrypting a file, and storing the header somewhere else.

The encrypted file would still require the offset for the header (currently 64 bytes, but #67 may change that), but the header can easily be written elsewhere.

This feature would be ideal for the most paranoid of users, who want to store their headers on cheap, destroy-able USB drives. Once the header is gone, that data will be gone for good.

This should be a fairly straightforward addition, but a pretty good one.

I plan to move to balloon hashing (with BLAKE3), instead of argon2id.

It just seems like a better, more suited hashing algorithm and I think now is a pretty good time to migrate over to it. Especially considering the new header style that is being planned (#67).

I tried to figure out what the -p (--password) flag for the encrypt and dencrypt subcommands is for, but could not find its use in the code. If we don't pass a key file or env, it always falls back to user input.

I found 1, 2, 3, 4 words with digits. Are they valid? Do they need to be removed?

I think now would be an ideal time to disallow Deoxys encryption.

Obviously decryption will be kept in place for backwards compatibility, and I don't plan to remove the option from the core library, but the implementation is a little slow in comparison to our other offerings. This is by no fault of our own, and no fault of the RustCrypto Team, but more due to the lack of good implementations in general. Deoxys is pretty new when you compare it to things such as AES-GCM, which makes it hard to create something optimized.

I do plan to re-add this functionality once the implementation matures a little.

In the next major release of Dexios, I think headers should be standardised to 48 (or 64) bytes at the start of the file. This would be able to account for the 16 byte salt, and the (current) max nonce length of 24 bytes, with room to spare.

The header would then be padded with random bytes/zeroes until it reaches that 48 byte length. This would mean there is no risk of data loss when stripping/dumping the headers, as the start of the encrypted data would always be after the 48 byte header.

We could possibly use these extra bytes to store information about the cipher and mode used (e.g. XChaCha20-Poly1305/AES-256-GCM and memory/stream modes). This would remove the need for the user to even care about what mode their file was encrypted with, and Dexios would be able to detect it automatically.

This would break compatibility with older files, so the major version would have to be incremented, in order to adhere to semantic versioning.

This will be done in conjunction with V4 Headers

Serialize argon2id parameters. This should also prevent the header version going up further for tweaks, as we can have ArgonParams::V1 and such.

This has been planned for a while, but I've been putting it off. I think it's finally an appropriate time to add it.

This is related to #35

I plan to zeroize every buffer that contains plaintext data, for cryptographic hygiene. From my benchmarks, this will have a negligible performance impact per block of data, but I feel as though the pros outweigh the cons.

This will suffice until we can figure out a way to implement what #35 specifies.

Hi !

At first ,thanks for Dexios ! It is a cool ,great job.

1. • Well ,it would be nice to integrate it into Explorer/context menu in Win (at least when decrypt) ,but i cant do it. An option is missing to tell Dexios to open a browser window for target folder designation. Can you do it ? Is it a viable idea ?
2. • ZSTD : I hope its not too late ,but a customizable compression setting/level would be nice ,too.
3. • Two weeks ago you rejected to add SERPENT into cascade mode. But now i please you again to reconsider it. Not for a permanent cascade mode ,just an additional option to use ,just like in P__oC___t. (It has no CLI ,drag and drop based ,i dont use it. Unconvenient for me)

All the best for you!

Hello!

Do you plan to add cascaded cipher support?
For example :xChaCha20-Serpent-Twofish? (256 bits)
IMHO its a pretty hard combo...

OTOH nowadays AES is a no-go.... Ok,its more than enough,but there are more secure algos,like above.
Overall personally i dont trust Rijndael....

I think a good starting point would be to add:

• English
• Russian
• French

(as a minimum)

This will be a very large task, and will require a lot of re-writing and translating.

I have a bit of knowledge regarding both Russian and French, but my translations likely will not be 100% accurate.

Hello !

Similar apps ,like yours ,PicoCrypt,etc,only supports ZIP-compression internally.
It would be nice to implement ZSTD...
What do you think about it?

Hello brxken128,
I would like to present you DexiosGui.

For fun, I have make some modifications to my crypto tool Cryptyrust for use it with Dexios format. For now, It only implement the "streamMode".
It is compatible with win/nix. With Light/Dark skin, config saving...

My Cryptyrust include big portions of Dexios code. I have included THX to you in the readme.

Suggestion: For the future, it will be excellent if Dexios was released in the form of lib. Separate the cli app and the core crypto (like my Cryptyrust) allow to use your format in multiple app.

Sorry for my bad english ;)

Thank for all your work on Dexios recently.

Update secret wrapper docs to mention that the only way to access the data is with the .expose() function, which means a simple search for expose() will show you wherever the key is used

Maybe add this to Auditing within the docs also

I have just realised that Dexios has not been adhering fully to the semantic versioning guidelines.

I've mistakenly forgotten to reset the patch version every time the minor version has been incremented - this will change when 7.5.0 is released (this update should include tar or zip functionality).

My apologies for any inconvenience.

Add a feature for auto-generating a key/passphrase.

This key could be completely random, 32/64 bytes

OR

It could be a passphrase, consisting of 2/3 large words and 4 digits at the end. I believe that's pretty memorable and offers enough security.

Provide a LUKS-style header, with room for keyslots.

This feature should be optional as it probably requires a fair amount of storage overhead (a few hundred bytes at most, but still noticeable)

A cryptographically secure 32-byte key should be generated.

The user's key is hashed, and that is used for encrypting the "master key".

This will allow for multiple keys to unlock the same file. security = strength of the weakest password

This would also require management functions, such as dexios keyslot add/del <file>

This builds on top of #67, and I think it'd be good to add support for changing the key (provided the header is V4).

This would be simple:

• Hash the user's old key
• Decrypt the master key
• Hash the user's new key
• Encrypt the master key with the new key

The documentation shows that ThreadRng is safer than StdRng::from_entropy

Unlike StdRng, ThreadRng uses the ReseedingRng wrapper to reseed the PRNG from fresh entropy every 64 kiB of random data as well as after a fork on Unix (though not quite immediately; see documentation of ReseedingRng). Note that the reseeding is done as an extra precaution against side-channel attacks and mis-use (e.g. if somehow weak entropy were supplied initially). The PRNG algorithms used are assumed to be secure.

Correct me if I'm wrong.