Track replication upstream.idle when calling on replica about vshard HOT 5 OPEN

Can't we lean just on box.info.status?

I assume you meant upstream.status?

No, but I got the answer :)

In my imagination tarantool should have some max_data_lag option to go into some 'stale data' state in such cases and broadcast the new state to clients.

Now, we are going to solve the same monitoring tasks in each client on the application level.

from vshard.

Can't we lean just on box.info.status?

NB: My old findings regarding replication monitoring: tarantool/doc#2604.

from vshard.

Can't we lean just on box.info.status?

I assume you meant upstream.status?
Still, no. upstream.status may be 'follow' but the replica still may have a huge lag to this upstream. For example, when master performs a batch of changes, which replica can't apply as fast as master did.

from vshard.

During fixing the initial problem i get the following upstream info in my case:

id: 1
    uuid: bbbbbbbb-bbbb-4000-b000-000000000001
    lsn: 60719
    upstream:
      peer: admin@localhost:48809
      lag: 0.0002281665802002
      status: stopped
      idle: 169.65149075794
      message: Can't modify data on a read-only instance - box.cfg.read_only is true

So monitoring upstream.lag doesn't solve the issue, and as @sergepetrenko mentioned upstream.status is not very effective too. Looks like it is best to check upstream.idle.

from vshard.

Looks like it is best to check upstream.idle.

It should be a combination of both lag and idle, whichever is greater. Or, as @Totktonada suggests, we may look at box.info.status == 'orphan', but then we have to make replicas go orphan if they see they stopped catching up with the master (this is a breaking change).

Or it can be box.info.status == 'stale_data'.

from vshard.