Comments (2)
Thank you for your feature request. We are working on moving CWL-Airflow to the cloud, so smooth integration with MWAA is on our TODO list.
If you can install custom python packages on MWAA, you don't need to swap the original airflow executable. You just need to install CWL-Airflow package and create your DAGs using CWLDAG class. CWL-Airflow uses the original Airflow and only extends its functionality with the ability to parse and execute CWL workflows. Let me know if you have any questions about installation and configuration of CWL-Airflow.
from cwl-airflow.
For others' reference, it seems one can install custom packages on MWAA: https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html
from cwl-airflow.
Related Issues (20)
- CWL-DAG Schedule bug HOT 4
- Use of Conditional workflows HOT 2
- Redirect CommandLineTool output to default task log file HOT 1
- Possible bug when uncompressing workflow content
- Question: Can i use airflow operators with CWL-Airflow? HOT 1
- Failure to install and run HOT 6
- cwl-airflow api error 404 issue HOT 1
- Getting 500 w/ "name 'os' is not defined" while trying to submit a job HOT 5
- Running CWL-Airflow with docker-compose does not work HOT 5
- CWLJobDispatcher, {task}, CWLJobGatherer not running after triggering with Airflow API HOT 2
- Change airflow dependency to less restrictive HOT 3
- BashOperator + CWLDAG HOT 3
- Getting standard error HOT 3
- Conditional Workflow HOT 1
- Getting standard output in airflow scheduler HOT 3
- CWL-airflow on AWS HOT 1
- Run workflow failed with dockerPull HOT 5
- run_as_user flag in default_args error HOT 1
- Feedback for CWL v1.2.1 requested.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cwl-airflow.