elasticdog / transcrypt Goto Github PK

It would be nice to be able to ask transcrypt what files are configured to be encrypted.

I'm not too sure how this could be fixed or if it's an issue with my setup, but if it's a small fix it'd be nice to have.

Happy to work on it if you can point me in the right direction :)

Steps to reproduce:

• Hard reset your working copy to HEAD^ (make sure you don't lose any local changes!)
• Make a change to encrypted file and commit
• Pull with rebase from a remote repo

Expected behaviour:

• Merge during rebase works as normal

Actual behaviour:

• During rebase, the diff is done against an encrypted version of the file

Using OpenSSL 1.1.1 (in my case, in Git for Windows, which uses OpenSSL 1.1.1a 20 Nov 2018), the clean script constantly leads to git commands printing out the following warning:

*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.

I can't use transcrypt in a directory that has spaces in its name.

For example, I have a directory named Source Code (aliased as src) in my home folder, and if I try running transcrypt in ~/src/foo (aka /Users/jtilles/Source Code/foo) then I get an error that resembles the following:

$(git rev-parse --show-toplevel)/.git/crypt/smudge: /Users/jtilles/Source: No such file or directory
error: external filter $(git rev-parse --show-toplevel)/.git/crypt/smudge failed -1
error: external filter $(git rev-parse --show-toplevel)/.git/crypt/smudge failed

I love the simplicity of transcrypt, but I cannot wrap my head around the current integration with gpg in #10 .

transcrypt --export-gpg <recipient> ends up saving the credentials of this repo encrypted for recipient under .git/crypt/<recipient>.asc. For reasons unknown to me, transcrypt --import-gpg <recipient> will look for .git/crypt/<recipient> or .git/crypt/<recipient>.asc or for the file <recipient> itself.

Before GPG support, it is clear to me that sharing access to encrypted files is done by sharing the cipher and the password (hopefully via safe communication methods).

But after GPG support, it is unclear to me why sharing access is done by sharing (manually that is) an encrypted message, rather than having the message committed into the repo (implicit sharing; the recipient wouldn't need to store the message since it will always be available in the repo).

How do you imagine the use of the GPG support? What happens if everyone flushes their credentials and they didn't keep a copy of the encrypted message? Default once again to the cipher and password stored somewhere, in a safe location?

That's what I can't wrap my head around. As it is now, GPG support is merely a built-in secure transport of the cipher and password, while I see it as an alternative to cipher and password, because anyone with access to the git repo, and access to one of the secret keys that can decrypt a member's encrypted message, gains access to the encrypted files.

The latter increases the attack surface, but it is more convenient. The former, well, it seems at least, add only overhead (considerable one going from share a password to setup gpg&co) with little gain (as long as the cipher and password are transmitted to the right people via safe communication methods).

Thanks in advance for the explanations!

Hi,

I am a graphic designer. I support the open source community with design contributions. I have made design contributions to different projects before. You can view my works from my profile. Would you like me to design a logo for TransCrypt?

Python supports objects in map key, but js doesn't. The js map will end up having the string "[object Object]" as key instead.

If implementing this has negative performance implications, then a pragma would be nice.

Objects with overridden +/- throw typeerror.

To reproduce:

import datetime
a = datetime.date.today()
b = datetime.date.today()
print(a - b) # throws typeerror in transpiled js

Defalult encrypt is always warning,plase add -pbkdf2 -iter 1024 to update

openssl enc -aes-256-cbc -md sha512 -pbkdf2 -iter 1024 -salt -in InputFilePath -out OutputFilePath

follow by https://crypto.stackexchange.com/questions/20941/why-shouldnt-i-use-ecb-encryption

As far as I can tell, there's no code path that updates the transcrypt scripts stored in .git/crypt, and these files are only updated on initialization.

This is problematic, as changes to these scripts are never propagated out to repositories with transcrypt already initialized.

In c8e4f72 the method for generating the salt was modified, and is not backwards compatible with the previous method. As a result, users who initialized a repository with transcrypt previous to this commit will be generating different salts, and thus be producing different output than those who initialized the repository after this commit.

For now, the work around is to have all users run transcrypt -u and rerun transcrypt, but this is a very tedious exercise to orchestrate across any organization of any size, and is prone to error.

transcrypt should have some facility to update these files automatically and transparently to users.

The most straightforward approach I can think of to fix this in transcrypt is to have it update these files every time it is run in a git repository, but that might be overkill.

A better solution might be to have these files be replaced with either symlinks or simple wrappers that call back out to transcrypt, and add this functionality in as a sub-module to the transcrypt program. This way these functions would always be tied in to the correct version expected by the current version of transcrypt.

As @perost-l14 mentioned in these comments on #78 transcrypt currently does some things that hinder its use for encrypting many and/or large files.

This ticket is to draw out suggested improvements so they don't get lost in the broader discussion in #78.

In particular, as paraphrased by me (@jmurty):

• encrypted files are Base64 encoded with the -a flag to openssl enc which takes up more space than binary data. This textual encoding may not be necessary since encrypted content isn't human-readable or diff-able as text.

• adding -delta to encrypted file lines in .gitattributes disables git packing (delta compression) to avoid wasteful work when pushing encrypted files that aren't compressible. Example:

  *.jpg filter=crypt diff=crypt -delta
  

Would it make sense to update transcrypt to use binary data instead of base64, and set or recommend -delta in .gitattributes by default?

What would be the implications of doing these things, for both new transcrypt'ed repos and existing ones?

You should be able to run the transcrypt command from anywhere within a repository, but it's currently only working from the top-level of the Git repo. Look into if this is related to the recent change in 0.9.5 or if it's always been that way and I've never noticed.

I clone my repo, and after running :
transcrypt -c aes-256-cbc -p 'thinkagain'
My encrypted files become modified.
So it seems : "Once transcrypt has stored the matching credentials, it will force a checkout of any exising encrypted files in order to decrypt them." -> that step is not happening.

I use the GitUp client on OS X, and I just discovered that it does not execute attribute filters (git-up/GitUp#224).

If I'm not careful (or someone else using GitUp isn't careful), it is very easy to stage and commit an encrypted file from GitUp.

Could transcrypt automatically configure a pre/post-commit or pre-push hook to ensure that any files that should be encrypted, are always encrypted?

Hey there.

I have a colleague who reports that their git status is never clean in the repos where we have encrypted files using transcrypt. We have compared versions of all the relevant software (transcrypt, git, openssl, OS...) and did not find any differences.

I am not sure why it is happening, but if you have any hints or advice on where to look to start tackling such an issue then I'm all ears!

I realise this is a bit vague and I don't have a way to reproduce the error so I understand if you aren't able to help but thought it was worth a shot.

When you merge branches containing transcrypt-encrypted files, if the files have conflicts the resulting "merged" file produced by Git for manual resolution ends up as a real mess, mostly or entirely full of the encrypted text which cannot be sensibly merged.

The root problem seems to be that git does not run the smudge/textconv filter on all the BASE, LOCAL, REMOTE conflicting versions of files before attempting a three-way merge, and instead attempts the merge using at least one – and maybe all – of the encrypted versions of these files. E.g. the encrypted content as available from git show :1:FILENAME and git show :2:FILENAME etc mid-merge.

I have not been able to find a workaround just within Git, and have instead added a custom "crypt" merge driver to handle this situation.

Here is a PR against our fork of Transcrypt as an example: ixc#1

This change isn't yet fully tested, and is written against a fairly old version of transcrypt.

Let me know if I should bring it up to date and submit a PR here directly to add the "crypt" merge driver to the latest version of transcrypt?

It seems that when I clone an existing repo and use transcrypt on it with correct information I end up rewriting all crypted files with an upgraded openssl.

transcrypt master and 9a8a1f4 tested
openssl-1.1.1c-2.fc30.x86_64 "OpenSSL 1.1.1c FIPS 28 May 2019"

git clone proto://server/my.git
cd my
transcrypt -c aes-256-cbc -p 'the-existing-phrase'

All git operations cause a rewrite of the crypted files. For example:

[damaestro@earth my]$ git status
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
[...] repeated for every file
On branch transcrypt-test
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   path/to/crypted.py
[...] all files listed
no changes added to commit (use "git add" and/or "git commit -a")

The files are locally readable.

It would be good to have GPG support that is one of the main benefits of git-crypt. Is there any chance of that?

I'm using codefresh to build Docker images (including the .git directory), with an entrypoint script that configures the git repo for transcrypt with a password from the environment, to decrypt secret files at runtime.

transcrypt thinks the git repo is dirty:

# transcrypt -c "${TRANSCRYPT_CIPHER:-aes-256-cbc}" -p "$TRANSCRYPT_PASSWORD" -y
transcrypt: the repo is dirty; commit or stash your changes before running transcrypt

When I run the git diff-index command that transcrypt uses internally (minus the --quiet arg), it seems to return some odd results:

# git diff-index HEAD -- | tail                                                                                                   
:100644 100644 3d6e0e93c2bfef67162836b2f0b5ad24c526929e 0000000000000000000000000000000000000000 M      staticroot/apple-touch-icon-precomposed.png        
:100644 100644 9089ef7ffea1efe2de79b49b5e6a5856360de4d6 0000000000000000000000000000000000000000 M      staticroot/apple-touch-icon.png                    
:100644 100644 7b74f8caf013ba164ca9049f7bd4eea0555ff720 0000000000000000000000000000000000000000 M      staticroot/favicon.ico                             
:100644 100644 5d00a90141a920a278e2a8239e560b7ea395a1db 0000000000000000000000000000000000000000 M      staticroot/humans.txt                              
:100644 100644 214e4119653f9c6a4c48cd0ebb06a6754f00f62b 0000000000000000000000000000000000000000 M      staticroot/robots.txt                              

But the repo is clean, cloned and built by codefresh:

# git status                                                                                                                      
HEAD detached at 153c57c                                                                                                                                   
nothing to commit, working directory clean                                                                                                                 

Strangely, after running git status, transcrypt suddenly agrees that the repo is clean and agrees to configure the repo:

# git diff-index HEAD -- | tail                                                                                                   

# transcrypt -c "${TRANSCRYPT_CIPHER:-aes-256-cbc}" -p "$TRANSCRYPT_PASSWORD" -y
The repository has been successfully configured by transcrypt.                                                                                             

Is there something wrong with the way codefresh is cloning the repo before it builds the Docker image?

Is there a more reliable way transcrypt can detect a dirty repo (as reported by git status)?

Doing a git diff I get to see the code differences ok but trying a "git add -p" shows only the encrypted content making it impossible to commit blocks of code.

Rekey fails when .gitattribute contains two star in path (**) and they are files with spaces in their name.

https://git-scm.com/docs/gitignore
A trailing "/**" matches everything inside. For example, "abc/**" matches all files inside directory "abc", relative to the location of the .gitignore file, with infinite depth.

There was also a unicode character in the filename, but I don't think that was the problem. (à)
I'm on windows.

.gitattributes:
/private/** filter=crypt diff=crypt

file:
the file.txt

Error:

	Proceed with rekey? [y/N] y

touch: unknown option -- N
Try 'touch --help' for more information.
error: private/the: does not exist and --remove not passed
fatal: Unable to process path private/the
***  rekeyed files have been staged  ***
*** COMMIT THESE CHANGES RIGHT AWAY! ***

In Git for Windows, currently in version 2.21.0 using Bash 4.4.23 and OpenSSL 1.1.1a, the salt generation in the clean script causes two issues:

a) The following warning is issued by OpenSSL everytime the openssl enc part of the clean script is called: hex string is too short, padding with zero bytes to length
b) salt generation differs between *nix and Windows due to line-endings.

Both issues have the same solution: make the salt consist of 16 hex characters not including any line-ending characters. Pull request on the way.

Although transcrypt's usage of the random number generator is limited, I've been thinking of reverting commit 0c288b2 even though it was originally done for portability. I need to explore what using transcrypt in a Windows environment realistically looks like, and if switching back would break that functionality (or if it could be worked around easily).

See these for reference:

• openssl/openssl#898
• An Analysis of OpenSSL's Random Number Generator
• How To Safely Generate A Random Number

I think it important to be more upfront about issues with rebasing like those brought up in #8. These limitations should be clearly stated in the README, so fools like myself don't almost push all their secrets in plaintext.

transcrypt's use of git rev-parse --show-toplevel in .git/config doesn't work with git worktree[1]. It appears, from my very limited testing, that git rev-parse --git-common-dir should be used instead. This works in both the linked and master worktree (in the top-level directories as well as subdirectories within each worktree).

[1] https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows

Currently this is not working, and will display the patches in the pager as encrypted text.

(frank)[280] oti@core-dev:~/Software/tesf  [master]  $   transcrypt -c aes-256-cbc -p 'foo bar secret'

The following configuration will be saved:

  REPO:     /home/oti/Software/tesf
  CIPHER:   aes-256-cbc
  PASSWORD: foo bar secret

Does this look correct? [Y/n] 


error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge failed 1
error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge failed
The repository has been successfully configured by transcrypt.

This also happens when trying git diff:

(frank)[284] oti@core-dev:~/Software/tesf  [master]  $ git diff
error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge failed 1
error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge failed

I just started using transcrypt recently and really like what it has to offer. One feature that I would really like to see is support for git archives. My particular use case is for Capistrano deployment to a remote server. Capistrano clones the git repository with the --mirror option to a local repo folder. It then calls "git update" to update the repo when needed. For deploying new releases it will use "git archive" on its local repo mirror and extract the output to a release directory. I would try to implement support for this myself but I am rather unfamiliar with the SSL encryption libraries and how Git filtering works.

The transcrypt script needs the column command to run. On newer Ubuntu systems, this is provided by the bsdmainutils package. It should be added to the requirements list.

The latest version of transcrypt calls mktemp without any arguments

tempfile=$(mktemp)

However on macOS Yosemite, and earlier, this is invalid syntax and will just cause usage information to be printed:

usage: mktemp [-d] [-q] [-t prefix] [-u] template ...
       mktemp [-d] [-q] [-u] -t prefix 

The newer man page for >= macOS 10.11 El Capitan says

     If no arguments are passed or if only the -d flag is passed mktemp behaves as if -t
     tmp was supplied.

So on the prior versions of macOS, it works to change tempfile=$(mktemp) to

tempfile=$(mktemp -t tmp)

However, that's not a drop-in fix because GNU mktemp will complain:

$ mktemp -t tmp
mktemp: too few X's in template ‘tmp’

So the invocation probably needs to be adjusted based on whether mktemp without args succeeds, in which case pass no args, otherwise pass -t tmp, to avoid breaking either way.

I keep getting:

error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge failed 1
error: external filter "$(git rev-parse --show-toplevel)"/.git/crypt/smudge faile

While running git diff

My attributes file includes:

#pattern  filter=crypt diff=crypt
manifests/hosts/*.pp filter=crypt diff=crypt
data/*.yaml filter=crypt diff=crypt

Oh, hi there.

Is there any way to use multiple transcrypt keys in the same repo? i.e. encrypt some files with key A and some with key B.

I can try to write some code if you point me in the right direction. Let me know! Thanks.

Hi there,

I set up transcrypt on a repository (let's call it repo A) the other day and everything was working fine. I set up transcrypt on a new repository (repo B) yesterday and that seems to be working fine as well but now when I started working on the repo A again I noticed that my files are not being decrypted locally.
Things are still working fine on repo B.

I am not sure why this is happening but I am happy to give you more information if you guide me on where to look!

Edit: Is it possible that a rebase could cause this issue?

Note to self, I need to better understand the failure/recovery scenario when there is a merge conflict happening with an encrypted file.

Hi.
I am using this on Windows 10 and after some issues, it is working. For it to work for others, I think my change should be put back into the tool.

First the two hints -

• Set core.quotePath to false (git config --add core.quotePath false). Without this Danish (and similar) characters are escaped to \octal and that breaks further processing of those files
• Next, to install transact on Windows without bash, I simply added it as a git alias and use the built-in bash interpretator in git. This is done with git config --add alias.transcrypt = !~/path/to/transcrypt/transcrypt. After this, I can install it with 'git transcrypt'.

And then the required fix in transcrypt itself.

The input file seperator (IFS) must be set to linefeed to avoid having the bash for statement split the input into words on space. Any files I have with spaces were not processed. So I added IFS=$'\n' before the "for secret_file in" loops (just before https://github.com/elasticdog/transcrypt/blob/master/transcrypt#L338)

Hope this change can get in.

PS. A great tool. Looking at using it for client-side encrypted backup.

transcrypt --version and transcrypt --help should also work outside git repositories, as they're frequently used to test whether the script is installed correctly. They give no output, though.

Example terminal session:

➜  /tmp mkdir test
➜  /tmp cd test 
➜  test transcrypt --version
➜  test git init
Initialized empty Git repository in /tmp/test/.git/
➜  test git:(master) transcrypt --version
transcrypt 2.0.0

Same applies to transcrypt --help.

Expected behavior: version also displayed outside an initialized git repository.

Any chance of getting a windows executable? (Actually I'm running it from minGW and works great, but would be nice to have a native version)

Indeed The example password indeed very secure, "correct horse battery staple"!

The reason is that default digest for the dgst and enc commands from MD5 to sha256.

When setting the digest to md5, the files can be decrypted properly again.

Rather then manually maintaining .gitattributes, transcrypt should have the ability to add and remove files in the repo similar to git-add. The remove should not delete files as git-rm does, rather stop encrypting the file and staging an unencrypted commit. What to do with the history would need to behave similar to a rekey.

Generally, at least on Linux, other users on a system are able to see the command line arguments given to a process. OpenSSL supports passing passwords via environment variables using -pass env:ENV_VAR_NAME which is not visible to other users besides root. I am not sure of a good fix.

One possible option which could generally apply to most other options would be to embed a short shell script within the git config an invoke it with something like this:

$(echo Zm9yIHcgaW4gaGVsbG8gd29ybGQKZG8KICAgICAgICBlY2hvICR3CmRvbmUK | openssl base64 -d | sh)

The main disadvantage being that it looks very shady.

I made a buildpack for heroku so it'll decrypt on deploy!

https://github.com/perplexes/heroku-buildpack-transcrypt

Regarding the statement:

There are much better options if your goal is to encrypt the entire repository.

Any chance you could give some examples of what you're referring to?

Thanks!

Steps to reproduce:

1. Clone transcrypt-enabled repo

2. Run transcrypt with valid credentials

3. Run git log -p (displaying all change sets, included encrypted ones)

4. Run transcrypt -f (flushing the credentials)

5. A. Run for object in $(cat .git/refs/notes/textconv/crypt); do git show $object; done (you'll see previously cached unencrypted data)

   or

   B. Run transcrypt with invalid credentials (encrypted data should NOT be available at this time)
   Run git log -p (you'll see previously cached unencrypted data)

I looked into this, and the reason this data is left unencrypted is due to the setting diff.crypt.cachetextconv=true. I image this has been done for efficiency/speed, but the result is unencrypted data left behind. I can think of two ways to handle this problem.

1. Make cachetextconv=true optional, defaulting to false. This might be the simplest fix. Today this is hard-coded to true.
2. Upon --flush, hunt down any cached data. This data can be found by inspecting <GIT_DIR>/refs/notes/textconv/crypt. That will contain a list of git objects that need to be removed. This is probably more difficult, but might be best because then caching could be used.

b'\0' produces invalid javascript.

Steps to reproduce

1. Create git repo
2. Initialize Transcrypt
3. Add Transcrypt rule to .gitattributes
   • empty-file filter=crypt diff=crypt
4. Add/Commit the .gitattributes file
5. Create the empty-file
   • touch empty-file
6. Add the empty-file

At this point, if you run git diff there will be an empty diff, instead of no diff. If you run git diff --exit-code and then check the exit code echo $?, you'll see it exits with a 1 (indicating a difference), instead of a 0 (indicating no difference). If you reproduce these steps except for initializing transcrypt and configuring the rule for the empty file, git diff --exit-code will exit with a 0.

From the user's perspective there's no indication what the problem is and it interferes with the ability to run any commands that depend on a clean working tree, eg git rebase.

in wsl , windows subsystem linux,it now work