BigQuery task decorated functions failing in Airflow 2.9.1 about airflow HOT 7 OPEN

Although I'm confused why this would affect @task decorated functions?

You are using BQHook, so it does not matter if task is decorated

Could this be something to do with updates to some of the google-api libraries between Airflow 2.8.4 and 2.9.1?

Can you downgrade the libraries and check if it fixes the issue?

@VladaZakharova - you might want to take a look at this one

from airflow.

Look like this is related to google-cloud-bigquery downgrading it to google-cloud-bigquery<3.21.0 fix for me

from airflow.

It is because the way we pass nowait param to insert_job method. When you are using @task then nowait is False but when you are using operator it is True

error coming from

I have drafted a PR to pin the lib version until we fix it #39583

from airflow.

I see. Thanks for the info.

from airflow.

Could this be something to do with updates to some of the google-api libraries between Airflow 2.8.4 and 2.9.1?

2.8.4

google-api-core==2.17.1
google-api-python-client==2.122.0
google-auth==2.28.2

2.9.1

google-api-core==2.19.0
google-api-python-client==2.127.0
google-auth==2.29.0

Although I'm confused why this would affect @task decorated functions?

from airflow.

@potiuk Yes, I guess my confusion is more about why hook is giving the error in this context rather than the operator; especially as it doesn't seen to be caused by the provider. I downgraded the libraries but the issue persists.

from airflow.

@pankajastro Interesting. Downgrading does work but I'm curious to know what issue is and why the hook method gives the error but the operator does not.

from airflow.