Pull with rebase doesn't handle encrypted files about transcrypt HOT 9 CLOSED

Thanks for reporting this...so the situation with git pull --rebase will only cause unexpected issues if the remote repo made a change to that same encrypted file that you are then rebasing against, correct? I haven't had time to mess around with this particular scenario yet (I'll try to take a closer look tomorrow), but wanted to point you in a direction that might help.

Essentially look at the MERGE STRATEGIES section of the git-pull man page. I suspect that the renormalize option to the recursive merge strategy might be beneficial in this case. I'll check back in after I've had a chance to explore things myself a bit more.

from transcrypt.

No worries, thanks for all your work on transcrypt! Yep that's correct - here is a bash snippet that will reproduce it:

#
# Set up our 'upstream' repo
#

mkdir crypt-test-upstream

cd crypt-test-upstream

git init .

transcrypt -p password -c aes-256-cbc -y

echo "Secret!" > sensitive_file
echo 'sensitive_file  filter=crypt diff=crypt' >> .gitattributes
git add .gitattributes sensitive_file
git commit -m 'Add encrypted version of a sensitive file'

cd ..

#
# Clone it
#

git clone crypt-test-upstream/ crypt-test-downstream/

#
# Edit upstream
#

cd crypt-test-upstream

echo -e "Another secret!" >> sensitive_file

git commit -am 'Changing upstream'

cd ..

#
# Edit downstream and pull with rebase
# 

cd crypt-test-downstream/

transcrypt -p password -c aes-256-cbc -y

echo "Another different secret!" >> sensitive_file

git commit -am 'Changing downstream'

git pull -r

If you run git diff at this stage, you'll see the diff isn't what you'd expect.

renormalize looks promising - I'll try that soon!

from transcrypt.

Thanks for the quick script to reproduce the issue. I honestly forgot, but I already set the merge strategy to use renormalize in the .git/config itself as part of transcrypt's initialization. The crux of the problem is mentioned in the gitattributes man page regarding merges with filters:

As long as a "smudge->clean" results in the same output as a "clean" even on
files that are already smudged, this strategy will automatically resolve all
filter-related conflicts. Filters that do not act in this way may cause
additional merge conflicts that must be resolved manually.

Right now, those properties are not true, so it's not able to automatically resolve all filter-related conflicts, which is why you get the raw encrypted diff along with the partial plain-text diff at the top. If I can think of a way to have the smudge filter just pass along STDIN if the decryption step fails (it already throws an exit code), the script could be updated so I think this will work. Give me a bit more time to think about this.

from transcrypt.

Changing the smudge filter (.git/crypt/smudge) to this:

#!/usr/bin/env bash
tempfile=$(mktemp /tmp/transcrypt.XXXXXX)
password=$(git config --get --local transcrypt.password)
cipher=$(git config --get --local transcrypt.cipher)
tee "$tempfile" | ENC_PASS=$password openssl enc -$cipher -pass env:ENC_PASS -d -a 2> /dev/null
[[ $? -ne 0 ]] && cat "$tempfile"
rm "$tempfile"

...seems to get things working as the gitattributes page suggests it should:

$ echo 'U2FsdGVkX18B9SN4DeiRUFIbTuYYLZZ7sX617msS6T5H36sM31sEqF9OHLvegVeD' | .git/crypt/smudge
Secret!
Another secret!
$ echo 'U2FsdGVkX18B9SN4DeiRUFIbTuYYLZZ7sX617msS6T5H36sM31sEqF9OHLvegVeD' | .git/crypt/smudge | .git/crypt/smudge
Secret!
Another secret!

I'm not sure if that will improve the rebase/merge issue, but I'll experiment some more. I don't like that it uses a tempfile, so I may try to use a named pipe instead to keep things in memory rather than use the disk. If you happened to encrypt some encrypted text and openssl doesn't fail on the second smudge, it may cause garbled text in the pipeline, but I'm not sure if that's a realistic concern or not, nor what the ramifications would be for the merge.

from transcrypt.

Hrm, so even with the improved smudge filter, I've not had any luck in getting this to work as desired. I'll have to think about this some more, but it might not be possible. Please let me know if you make any headway.

from transcrypt.

I have a slightly different clean filter as well to have clean -> clean output the same text:

#!/usr/bin/env bash
tempfile=$(mktemp /tmp/transcrypt.XXXXXX)
password=$(git config --get --local transcrypt.password)
cipher=$(git config --get --local transcrypt.cipher)
tee "$tempfile" | ENC_PASS=$password openssl enc -$cipher -pass env:ENC_PASS -d -a &> /dev/null
if [[ $? -ne 0 ]]; then
    filename=$1
    salt=$(openssl dgst -hmac "${filename}:${password}" -sha256 "$filename" | tail -c 16)
    cat "$tempfile" | ENC_PASS=$password openssl enc -$cipher -pass env:ENC_PASS -e -a -S $salt
else
    cat "$tempfile"
fi
rm "$tempfile"

...and then I also tried setting the attribute to:

sensitive_file  filter=crypt diff=crypt merge=union

...and then finally pulling with:

$ git pull --rebase -s recursive -X renormalize

...which gets things close, but not quite 100% there.

from transcrypt.

Okay, one more experiment update...no need to change from the original clean/smudge filters at all, just try this out:

$ git pull --rebase -X theirs

...and then look at the history (git log -p), it preserves the changes that upstream did and then commits your downstream changes as the eventual correct version. Not perfect since you can't decide to keep parts of both during the rebase, but the history is there and clean, so you could edit the sensitive file at that point. You would also have to check the sensitive file's history specifically for changes, but it's an option.

I'll keep playing around to see if I can come up with something better. I'm honestly not sure why the regular recursive merge with renormalize doesn't dump things in plain text.

from transcrypt.

Hey @elasticdog -- I've also come across this issue with no luck resolving, If you need a test case, I've created https://github.com/nicktacular/transcrypt-rebasing which, you can do this to replicate:

$ git checkout branch-B
$ git rebase -p master

My current workaround is to manually resolve in plaintext in each branch in a new file and use the plaintext content of that new file to resolve during the rebase.

from transcrypt.

I've toyed with so may incantations of merge options and still don't have a great solution, but it looks like this is a known issue:

https://git.wiki.kernel.org/index.php/SmallProjectsIdeas#A_way_to_check_merges_using_textconv_filters

...note that you can avoid some manual editing if you use git checkout --ours <file> or git checkout --theirs <file> when resolving the merge conflict. Beyond that, I don't know if there's much we can do here with how the textconv filters are currently implemented and utilized for merges, as Git's behavior makes sense for normal binary files, but not really in the context of files having the actual editable data after the smudge filter is applied.

I think I'm going to close this issue for the time being, but will certainly think about it as future Git versions come out and offer possible solutions.

from transcrypt.