Infinite option for bigQueryRetry about kafka-connect-bigquery HOT 3 OPEN

@aakarim, what do you mean by it's capped . I think only condition is minimum 0. If you set it to a large value, doesn't it retry?
Could you check if setting bigQueryRetryWait and bigQueryRetry both to large values help ?

from kafka-connect-bigquery.

@b-goyal Yes it does retry, it just only retries set number of times. I'd like it to keep retrying until the quota has expired (usually after a day). I could multiply it out, so if I have a bigQueryRetryWait of 15s that means that I need to retry 5,760 times max before giving up, but I'd like the flexibility to just set it as infinite and have it keep retrying forever.

from kafka-connect-bigquery.

bigQueryRetry is of type int so it is basically capped at 2147483647.
The bigQueryRetryWait is defaulted to 1000ms ( 1sec) . Doc link : https://docs.confluent.io/kafka-connectors/bigquery/current/kafka_connect_bigquery_config.html#:~:text=Importance%3A%20medium-,bigQueryRetryWait,-The%20minimum%20amount
If you set bigQueryRetry to the highest value possible which is 2147483647, with 1sec wait before retry, you will be retrying for 68 years.
I believe we do not need any explicit value to make the connector retry for days, if we set it to max, it should serve the purpose.
Does this help?

from kafka-connect-bigquery.