re-running failed operations with TzKT backend is so slow, it hangs about tezos-reward-distributor HOT 2 OPEN

An update on this.

When running in retry mode, for every delegator with a failed payout, we update the balance:

https://github.com/tezos-reward-distributor-organization/tezos-reward-distributor/blob/master/src/pay/retry_producer.py#L106-L107

but the only other occurence of update_current_balances() function in the producer logic is in CalculatePhase4 which is for founders/owners.

We are not actually querying individual delegators balances with tzkt API during initial payment: we are bulk querying the indexer with a balance list of delegators self.reward_provider_model.delegator_balance_dict.

Only during retries, we do it, which takes a very long time and often fails.

A solution would be to modify the retry logic to query all balances again using the same API as initial payment. I'm not going to do this now, instead I'll just record my observations here.

from tezos-reward-distributor.

This is what's in the verbose log when this happens:

2024-02-15 17:46:15,839 - producer  - DEBUG - Requesting https://api.tzkt.io/v1/accounts/tz2xxxxx
2024-02-15 17:46:15,950 - producer  - DEBUG - Response from TzKT is:
{'activeRefutationGamesCount': 0,
 'activeTicketsCount': 0,
 'activeTokensCount': 60,

from tezos-reward-distributor.