BigQuery: raise a custom exception if `400 BadRequest` is encountered due to "internal error during execution" about python-bigquery HOT 14 CLOSED

At this point, going to close this item as "Will not fix" due to competing priorities.

from python-bigquery.

it is not possible to retry the job from within the library

@pietrodn It is possible to retry the job from within the library, for that need to use client.create_job method which creates a new 'job ID' on every retry.

from python-bigquery.

@tswast I'm pretty sure that this is a back-end issue: the API shouldn't be returning '400 Bad Request' for internal server errors. Can you confirm?

from python-bigquery.

@shollyman Is this related to another BigQuery backend rollout?

from python-bigquery.

I agree that 400 Bad Request is the wrong response code for this error.

from python-bigquery.

I've filed bug 124319762 internally to track this issue. I see several similar reports internally, so this is likely not new behavior.

@bencaine1 If you have a support plan, I recommend filing a ticket with them to raise the priority of this issue on the BigQuery backend.

from python-bigquery.

@tswast Do you want to leave this issue open (i.e., do you imagine we will be making changes here to work around the 400?).

from python-bigquery.

Let's close this. The client workaround would be to look for certain text in the response body and ignore the response code, which I'd prefer not to do if we can avoid it.

from python-bigquery.

I saw this error again today. Has the underlying BigQuery issue been fixed? Is there another new issue with the same symptom?

If this bug continues to recur with various BigQuery bugs, I think there is (sadly) a case for having the client retry, because otherwise the non-Google customer application becomes responsible for retrying. That seems ... worse.

from python-bigquery.

Backend issue was closed as infeasible. Backend engineers say:

Connection error is the only retryable error returned as a job result, so you only need to add the logic to retry jobs.insert() if jobs.getQueryResult() returns 400 and the error reason set to the "jobBackendError" (which means that the job failed with the connection error), but I don't think you can reuse the job id for this case.

Since the job ID cannot be reused, this error is one that requires the whole job to be retried from the beginning. I think it's reasonable to do this, though will likely a bit difficult to do, as the failure won't be discovered until .result() is called.

from python-bigquery.

If the error reason is "jobBackendError", it should be definitely be included in the BigQuery error table, so that developers can deal with it appropriately.

from python-bigquery.

If the job ID is not reusable, and it is not possible to retry the job from within the library, google-cloud-bigquery could catch this BadRequest exception and re-raise is as an InternalServerError exception, so that the application code using the library can easily retry the whole class of server errors transparently.

from python-bigquery.

google-cloud-bigquery could catch this BadRequest exception and re-raise is as an InternalServerError exception,

I think this is a reasonable feature request.

from python-bigquery.

Some requirements for a custom exception:

• Backwards compatible -- inherit from the Google API error base class
• Preserves stacktrace -- use the exception wrapping mechanism to preserve all the context from the original exception.
• Clear that it's a BigQuery-related custom exception from the name. Example: BigQueryServerError.
• The message of this exception includes a code snippet on how to recreate the query job (to retry it) with Client.create_job.

from python-bigquery.