In the next video, you will be formally introduced to Apache Airflow and Ajay will discuss some of its distinguishing features.
In the above video, we saw the following feature of Apache Airflow:
- Airflow was started in October 2014 by Maxime Beauchemin at Airbnb.
- It was built to address the company’s increasing complex workflows or data pipelines and monitoring needs.
- It is a tool that can programmatically create, schedule and monitor data pipelines.
- Data pipelines in Airflow are:
- Implemented in the form of Directed Acyclic Graphs (DAGs);
- Created using Python code; and
- Can be generated dynamically.
Following are some of the features of Airflow discussed in the video.
In the next segment, you will learn about a very important concept in Airflow known as DAG.
Additional Reading
Official Airflow website: This is the go-to guide for anything-everything related to Airflow.
Apache Airflow GitHub: In case you are interested in the source code of Airflow itself.
Report an error