Code Monkey home page Code Monkey logo

nix-airflow-template's Introduction

nix-airflow-template

Airflow local instance template, with a help from Nix and Poetry. Use it to get acquainted with the platform.

Environment

To establish a local environment from this repository, you will need Nix installed

Once Nix is installed, simply run:

nix-shell
make init

What is inside?

  • local non-containerized Postgres instance;
  • latest Airflow instance (as of writing this readme);
  • Python 3.7 runtime;
  • version-pinned with Poetry;
  • shortcut commands to control the instance.

Instantiating Airflow

make airflow-start

How to test it

Navigate your browser to localhost:3333, enable the example DAG, refresh the page and observe that the DAG has been scheduled for execution. In a few seconds you will see that it completes. On the rightmost column, click the "Tree View" icon.

Every green square represents a dag task, every circle is a complete DAG instance. You can find the result of the operator by clicking on a square, and selecting View Log -> XCom.

How to reset everything

The following command removes the current Airflow DB instance, and recreates it from scratch again

make reset

How to stop the instance

To stop Airflow and Postgres instances run

make airflow-stop

License

0-clause BSD License

nix-airflow-template's People

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.