celery_executor got an unexpected keyword argument 'remove_running' about airflow HOT 12 CLOSED

Marked them to be included in 2.9.2rc2

from airflow.

Hey @vincbeck @o-nikolas -> looks like we got it now when 2.9.2rc1 is using the old celery provider.

Way to reproduce:

1. Run this:

breeze start-airflow --use-airflow-version 2.9.2rc1 --load-example-dags --load-default-connections --executor CeleryExecutor --airflow-extras celery

2. Check installed packages:

root@c45830039533:/opt/airflow# pip freeze | grep celery
apache-airflow-providers-celery==3.7.1
celery==5.4.0
root@c45830039533:/opt/airflow# pip freeze | grep apache-airflow
apache-airflow==2.9.2rc1
apache-airflow-providers-celery==3.7.1
apache-airflow-providers-common-io==1.3.2
apache-airflow-providers-common-sql==1.14.0
apache-airflow-providers-fab==1.1.1
apache-airflow-providers-ftp==3.9.1
apache-airflow-providers-http==4.11.1
apache-airflow-providers-imap==3.6.1
apache-airflow-providers-smtp==1.7.1
apache-airflow-providers-sqlite==3.8.1
root@c45830039533:/opt/airflow#

3. Connect to http://localhost:28080/

4. Run "example_complex" dag

5. You will see BaseExecutor.change_state() got an unexpected keyword argument 'remove_running' airflow_scheduler - errors. I am afraid we will need to to handle back-compatibility there.

from airflow.

#40035 (this one is just a unit test, but it links to the other issues and PR that fixes it as wel. That was merged a couple days ago)

from airflow.

@o-nikolas -> but I think the problem is that the new executor interface does not work with old celery provider.

from airflow.

(or the other way round - maybe the celery executor 3.7.1 does not work with 2.9.2 - and likely 2.8, 2.9 in general - not sure as I am on a meeting today).

from airflow.

I'm AFK right now, I can look later, but the traceback seems to be base exec being called with a param it doesn't expect (and the type error is now caught correctly as of a couple days ago)

TypeError: BaseExecutor.change_state() got an unexpected keyword argument 'remove_running'

from airflow.

Ok. I see those were not marked for 2.9.2 so they should be in this case.

from airflow.

I marked it to be in 2.9.2 but it got removed :( But anyway, yes #40012 is the fix to this issue

from airflow.

Actually - it's not a full fix. It will only work with "fufure" celery provider when it is released. The older versions of celery providers will not work with it @vincbeck @o-nikolas. I guess that means that we

1. have to wait with 2.9.2 until new celery provider is released
2. have to yank celery provider 3.7.1 because #39212 essentially broke the executor for all past versions of Airflow, where #40012 fixes it in the future celery provider

cc: @eladkal @ephraimbuddy

from airflow.

This might actually justify slightly acceleratte celery /k8s provider voting, because we won't be able to effectively test 2.9.1rc1 until celery/k8s providers are released cc: @eladkal .

from airflow.

1. Celery provider 3.7.2rc1 will be released before Airflow 2.9.2
2. Celery provider 3.7.1 is yanked

I guess we can close this issue?

from airflow.

Yep.

from airflow.