[NEW] Atomic slot migration HLD about valkey HOT 5 OPEN

I have an implementation. Didn't write it down yet, but I recorded a song which explains it all. Please review. https://suno.com/song/b06c2a5b-3760-4916-9f56-eb3fe66f24e2

from valkey.

@madolson and I discussed another option (in the context of atomic slot migration) which essentially implements slot-level replication. Here is the high level flow

1. source (parent) forks a child with a set of slots to migrate
2. child streams slots to be migrated in the RDB format (same as replication)
3. target needs to support no-blocking load of the RDB stream (think of co-routines)
4. source (parent) starts capturing updates in the migrating slots right after forks
5. child completes streaming
6. source (parent) pauses clients writing to these slots
7. source replicates captured updates to target
8. source starts the slot ownership transfer process (depending on cluster v1 vs v2, we could take different paths)
9. source unblocks paused clients with -MOVED <target>

any failure on the target before completing step 8 would abort the migrating process on the source.

from valkey.

I have an implementation. Didn't write it down yet, but I recorded a song which explains it all. Please review. https://suno.com/song/b06c2a5b-3760-4916-9f56-eb3fe66f24e2

Are you referring to #298? I will take a look next but I don't see it solving this issue?

from valkey.

@PingXie The song is about atomic slot migration. #298 is about subscribing to slot migrations. Not the same.

from valkey.

@PingXie The song is about atomic slot migration.

Ship-it

from valkey.