Error while pulling from MegaSync about lfs-folderstore HOT 11 CLOSED

So the only way this could happen is if creating the temp file failed because it existed, as the error suggests; from the code:

	// Copy to temp, since LFS will rename this to final location
	// Use git dir as base to ensure final path is on same drive for LFS move
	dlfilename := downloadTempPath(gitDir, oid)
	dlFile, err := os.OpenFile(dlfilename, os.O_WRONLY|os.O_CREATE|os.O_TRUNC|os.O_EXCL, 0644)
	if err != nil {
		api.SendTransferError(oid, 5, fmt.Sprintf("Error creating temp file for %q: %v", filePath, err), writer, errWriter)
		return
	}

There shouldn't be any reason it would exist though, because it's only there temporarily until git-lfs moves it into the final position in your repo. Once that's happened, it'll never be requested for download again so wouldn't come up in the task list. So we know that something is creating that temp file more than once. There are a few possibilities:

1. The git-lfs move part didn't work the previous time.
2. The download didn't finish the previous time (out of disk space, transfer error - although I'm pretty sure in that case the temp files are cleaned up
3. More than one git-lfs fetch command is executing at once

It's hard to know which of these is the case. Here's a couple of possibilities to check first:

1. Check whether F:\MegaSyncLOTTS\80\0e\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8 is the correct file. Do this by just copying it somewhere else and renaming it so you can open it in an editor. It's possible that if it's corrupted, your co-worker isn't re-pushing it because their git thinks it already has, but when yours downloads it it checks the SHA and finds it to be incorrect, which abandons the "move" part of the process, leaving the temp file there (which then leads to this error). If it is, the resolution is to fix that file from your coworker's source by deleting it there and manually pushing it using git lfs push --object-id origin 800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8

2. Check whether you already have a copy of YOUR_REPO.git\lfs\objects\80\0e\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8 that doesn't look right. If so, delete it as well as the contents of .git\lfs\tmp before pulling again.

from lfs-folderstore.

Thanks for the reply.

Which SHA sum is used here? Can't find any shaXXXsum that returns a correct value here...

I copied the file into an other project, renamed it to file.uasset and opened the project, everything is fine with the file.

I did not already have a file in the specified folder either. I'm still lost here.

I'm not sure I said this the first time, but I tried to re sync the Mega folder on another driver, re clone the repo from scratch and I still get the same error.

As for the two concurrent lfs here is a ps from the pull:
`1840 828 1840 30400 cons0 197609 22:55:04 /mingw64/bin/git

1975 1 1975 33372 ? 197609 22:55:11 /mingw64/bin/git-lfs`

from lfs-folderstore.

Git LFS used SHA-256 so yes would be worth checking that's the correct SHA (although it shouldn't get out of whack unless someone edited the file directly on the file share)

from lfs-folderstore.

Thanks. Any other ideas? I've checked everything I can think of and your leads too.

from lfs-folderstore.

I checked all the file with the sha256sum, everyone is OK.

from lfs-folderstore.

I have no explanation for this, it should be impossible for that temp file to already exist if you cleaned it out beforehand, because the file should only be requested once. The only thing that stands out to me is that you're using MINGW and as a result a rather odd path syntax (lfs.customtransfer.lfs-folder.args=/F/MegaSyncLOTTS/), I'd encourage you to try it under a regular command prompt instead and use lfs.customtransfer.lfs-folder.args=F:/MegaSyncLOTTS to eliminate that variable.

You could potentially fix this manually if you've checked the SHA and just copy F:\MegaSyncLOTTS\80\0e\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8 to F:\UE Project\lotts.ter.git\lfs\80\0e\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8, which would at least let you continue (assuming it's just that one file). I realise that won't fix clones though. I'd suspect something odd like symlinking in your local filesystem but if it's happening with other clones I have no idea.

If it is that one file, you might be able to "fix" the clone by making a minor edit to that file and re-committing it (generating a new hash). All LFS objects are just binary data so in theory there should be no difference from one to the other, but if this has just started happening it suggests something odd about that particular file, although what that might be I can't imagine. This plugin is literally just copying files around (when pulling, into a temp dir so that core git-lfs can move it into its final location - it's like this because it's emulating a download) so there's not a lot that can really go wrong unless something is being weird in the filesystem.

from lfs-folderstore.

Hmm actually picking through the log again I'm noticing that git-lfs is requesting this file twice, in 2 different batches. In the first batch, which is embedded in with a lot of other files, it completes OK:

21:09:29.080820 trace git-lfs: xfer[lfs-folderstore]: Received download request for 800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8
...
21:09:29.133779 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8","bytesSoFar":240555,"bytesSinceLast":240555}
21:09:29.138763 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8","path":"F:\UE Project\lotts.ter\.git\lfs\tmp\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8.tmp"}

Then later on it's requesting the same file again, in a new batch (restarting the folderstore adapter presumably before it's moved the existing download into place):

21:09:29.255978 trace git-lfs: filepathfilter: accepting "Content/RLContent/Alana/Alana_fbm/Std_Eye_R_Pbr_Normal.uasset"
Downloading Content/RLContent/Alana/Alana_fbm/Std_Eye_R_Pbr_Normal.uasset (241 KB)
...
21:09:30.414245 trace git-lfs: xfer[lfs-folderstore]: Received download request for 800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8
21:09:30.415169 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8","error":{"code":5,"message":"Error creating temp file for "F:\\MegaSyncLOTTS\\80\\0e\\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8": open F:\UE Project\lotts.ter\.git\lfs\tmp\800e187a618a9975f9b4f9761b8797ea997575bb72bc98d611b16dbc13d01dd8.tmp: The file exists."}}

It should not be requesting the same content twice in one session, I've never seen it do that - but that would explain why the temp file was already there the second time. Copying the file manually will definitely fix it (git-lfs won't need to request it at all then) but this looks like either a bug or a change in behaviour in git-lfs.

from lfs-folderstore.

OK I highly suspect this is a new git-lfs bug (it would cause duplicate downloads elsewhere) but since I haven't worked on git-lfs in years I don't have the familiarity to fix it right now, so I've just worked around it in the transfer agent. Not ideal, but it should work and in this case there's no extra download overhead anyway.

https://github.com/sinbad/lfs-folderstore/releases/tag/v1.0.1

from lfs-folderstore.

I'll try this right away

from lfs-folderstore.

Okay, it's working like a charm if I use WSL and linux binaries.
It fails when I use git for windows on a hash that is not correct... It seems that sha256sum.exe and sha256sum on WSL do not gives the same output for the same file. I'll look into this on my side, does not seems to be related to lfs-folderstore now.

A big thanks for your help on this one!

If this case help you raise a case to git-lfs I still have the old binaries and will be able to reproduce this in the future I think.

from lfs-folderstore.

I'm glad it's resolved your issue, however the sha256sum exe shouldn't be a factor, since git-lfs uses the Go library version of SHA which has to be identical on all platforms, otherwise repositories wouldn't work across platforms. The lfs-folderstore tool never calculates any SHAs, it just responds to requests from git-lfs for specific SHAs. So I'm not sure what's going on with your git for windows errors, but you're right that it's likely unrelated, so I'll close this issue.

from lfs-folderstore.