send snapshot size about raft HOT 8 CLOSED

The snapshot should contain the full state of your state machine, which in turn should be normally kept in-memory, so generally the constraint of having the snapshot fit in memory should be acceptable my most applications. What use case do you have?

This raft library could be changed to perform multiple RPC requests for sending a snapshot, but that's a bit out of scope right now, since we didn't have that need yet for our application. Patches would be welcome tho.

from raft.

if i use this raft lib to replicate my RocksDB database, and snapshot of RocksDB may bigger than memory size of machine. So the database snapshot can't be send by one request.

from raft.

I see. As I said, sending snapshots in multiple chunks is currently not supported nor planned. I would have no objection if you wanted to do that work as contribution, but at the moment I don't have time to work on this myself.

from raft.

have some detail design? and i can contribute some patch.

from raft.

have some detail design? and i can contribute some patch.

I don't, as it's not going to be trivial. The main issue is that if your snapshot doesn't fit in memory, then your finite state machine will also presumably not fit in memory, correct? That means several changes in various parts of the library. In general Raft seems to be a better fit for state machines that fit in memory, and that's why many libraries what there make that assumption. It's possible to have your state machine partly persisted on disk, but that complicates the design and the library. One cheap option would be to rely on OS caching/swapping, but not sure if that would be acceptable for your use case.

from raft.

have some detail design? and i can contribute some patch.

I don't, as it's not going to be trivial. The main issue is that if your snapshot doesn't fit in memory, then your finite state machine will also presumably not fit in memory, correct? That means several changes in various parts of the library. In general Raft seems to be a better fit for state machines that fit in memory, and that's why many libraries what there make that assumption. It's possible to have your state machine partly persisted on disk, but that complicates the design and the library. One cheap option would be to rely on OS caching/swapping, but not sure if that would be acceptable for your use case.

ok, thanks.

from raft.

Can I close this?

from raft.

Can I close this?

yes

from raft.