Reverting to older files or state is troublesome about mirror HOT 9 CLOSED

that delete operation on the remote outdates the mtime of the old file

Yes, that is what happens, mirror creates a fake mtime of the delete to be able to tell which local vs. remote version "wins".

I think this is necessary for at least how mirror is currently fundamentally designed, but there is a slight possibility that, after deciding which version has won (and the delete wins), mirror could be better about cleaning up the marked-deleted path, e.g. by unsetting the fake mtime.

I'll poke around at that when I get a chance.

from mirror.

In the mean time, so that I can understand and workaround - is that fake mtime dropped when mirror is restarted or does it persist in some way? So if I restart mirror, can I take it that the state is reset? Should I need to restart mirror just on the client or on the server as well?

from mirror.

Yep, it's only kept in memory, so restarts will reset it.

from mirror.

Both the server and client have their own copy of it in memory, so would need to restart both.

from mirror.

mirror could be better about cleaning up the marked-deleted path, e.g. by unsetting the fake mtime.

Well, that would take care of one use case but beware with that, though. What if there was another spoke on third device - say a laptop that won't connect for a couple of days, but is expecting the latest versions - but not necessarily the latest mtimes) to be synched up when it connects? That's the scenario I was hoping to use mirror for so I can dev in the field and not have to worry about, "Did I remember to git push at home yesterday?"

from mirror.

What if there was another spoke on third device - say a laptop that won't connect for a
couple of days, but is expecting the latest versions

Ha, right. Although that is also what happens if you were to stop mirror and then reconnect... It forgets the fake mtime and would treat the missing-for-a-few-days laptop's file as "well, this must be good".

Which, in general is a safe default, if either server/client has a file, go ahead and keep it, with the assumption that the user re-deleting something is less painful than un-deleting something they wanted to keep.

Dunno, will think about. Ideas appreciated of course.

from mirror.

Just thinking it though I would love my server to say "here's a replay of all the transactions since you last connected so we know you have the right files" - How confident would that sound? Whether it takes that replay from the ext4 journal (how far back does that go?) or maintains its own journal (with user-defined expiry time/history length) for nominated folders that the server keeps a track of. Just a thought - hope I haven't ruined your plans!!! Keep up the good work! It's at least usable in this state now that I know what to look out for!

from mirror.

Ha, right. Although that is also what happens if you were to stop mirror and then reconnect... It forgets the fake mtime and would treat the missing-for-a-few-days laptop's file as "well, this must be good".

Is that right? Didn't you point out that the fake mtime is also maintained on the server, though, and a stop-start both sides would be required to flush it?

Both the server and client have their own copy of it in memory, so would need to restart both.

Furthermore - what happens if a third device maintains a copy of the fake mtime, while the first 2 devices are stop-started?

from mirror.

stop-start both sides would be required to flush it

Right, sorry, I meant "stop both sides".

third device maintains a copy of the fake mtime

Also right, "stop all sides". :-)

here's a replay of all the transactions since you last connected so we know you have the right files

That is not a bad idea conceptually. Pragmatically, the biggest challenge is clock drift, e.g. even with mtime, if the server/client get out of sync, a "but I just wrote that!" on a slightly-behind client can get stomped by a slightly-ahead server (or vice versa). (mirror has an extremely extremely (initial on connection) naive clock drift detection because I had a few issues reported by users, and experienced it myself, that ended up being due to that.)

So, just a clock by itself is hard/"impossible" to get a true combined ordering of "this then that happened" across several remote machines (which is what the journaling approach would need to do, e.g. interleave "server thought this happened" with "client a thought that happened" with "client b thought something else happened" and come up with a sane result).

Granted, there are ways to do this, e.g. lamport timestamps (probably, I've not actually used them before) and other distributed computing primitives, but that gets into "building a real distributed system" instead of "just compare some mtimes and pick one". :-)

I might be able to detect the condition locally when a "this path in memory has mtime of fake+1 and oh wait the file system event told us it just came back" and then bump its original mtime (which mv didn't bump) to be fake+1+1. :-) Granted, that's messing with your mtimes, but hopefully only slightly, e.g. by a second or two, just enough for it to be newer.

from mirror.