Comments (5)
I see the case for a branch operator, though we tend to wire things down different path by using an attribute of the data, and an empty batch typically never happens except if there is a problem.
An easy implementation would be to essentially derive the PythonOperator
into a BranchPythonOperator
, and have the assumption that the python_callable
returns the name of the branch to take, namely the task_id of the direct downstream task to follow. Then this operator would set a failed status on all the directly downstream task_ids not returned.
from airflow.
I wrote a quick sketch of what it may look like in this branch https://github.com/airbnb/airflow/compare/branch_python_operator?expand=1 but haven't tested it, let me know what you think.
from airflow.
from airflow.
Folks, when are you planning your next release. I would like to use this in our staging environment and that is currently deployed using "pip"
from airflow.
v1.2.0
is out!
https://pypi.python.org/pypi/airflow
I pushed the new version of the docs to http://pythonhosted.org/airflow/ also
from airflow.
Related Issues (20)
- airflow hashicorp vault integration - path with subfolder is not wokring
- Status of testing Providers that were prepared on April 10, 2024 HOT 20
- Remove extra slash from update_job_permission api HOT 1
- Two buttons to reset password for an airflow user HOT 4
- sla_miss_callback is not working with 5 input parameters as official document, only dag_id 1 input is allowed, no enough context for alert HOT 4
- on_failure_callback is not called when task is manually marked as failed HOT 8
- Components stuck waiting on migrations HOT 2
- You cannot see newer DAG runs if you open a link to a specific DAG run [UI Bug] [New as of 2.9.0?] HOT 3
- Unable to mock Variable in python unittest HOT 3
- [BACKFILL] We shouldn't be waiting for a whole batch of dag runs to be finished before triggering new ones
- Flaky MappedTaskUpstreamDep test `test_mapped_task_upstream_dep` on Postgres
- add blob size to LocalFilesystemToGCSOperator HOT 1
- GlueJobOperator with local script location fails on consecutive runs HOT 1
- Control DAG Executor Assignment through RBAC HOT 5
- Tasks are in queued state for a longer time and executor slots are exhausted often HOT 12
- Mismatching dates in Run ID and Grid View HOT 1
- Flaky docker example test around `mpi4py` builds
- Allow searching dag by display name HOT 1
- Status of testing Providers that were prepared on April 13, 2024 HOT 7
- Allow passing airflow params as job parameter in databricks job
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.