Welcome to this new module on ’Building automated pipelines with Apache Airflow’.
Let’s hear from our SME about everything that you will cover as part of this module.
As Ajay mentioned in the previous video, in this module we will learn about Apache Airflow and how to schedule, automate and monitor complex data pipelines by using it.
This module is divided into three sessions.
In the first session, you will learn about the following:
- Data pipelines and its orchestration
- Introduction to Apache Airflow and it’s features
- Important concepts and internal architecture
In the second session, you will learn about the following:
- How to set-up Airflow
- Discuss Airflow configurations and a CTL walk-through
- User Interface (UI) walk-through
- Airflow operators and some hands-on demo
Finally, in the third session we will learn about the following:
- Orchestrate a real-world problem statement using Airflow
- Discuss advanced concepts of Airflow
Structure of This Module
Session 1 – First mandatory session – Introduction to Apache Airflow
Session 2 – Second mandatory session – Hands-On with Apache Airflow
Session 3 – Bonus content for session 2 (Optional)
Session 4 – Third mandatory session – Real-World Use Case of Airflow
Session 5 – Bonus content session 4 (Optional)
This Module has three mandatory sessions (Session 1, 2 and 4) and two bonus sessions (Session 3 and 5). It is advised that you attempt bonus sessions only after you have finished the mandatory ones or if you have enough time to do so.
Guidelines for This Module
This module is a mix of theory and demonstration as follows:
Session 1 – Theory
Session 2 – Theory + demonstration
Session 3 – Bonus Content(Theory)
Session 4 – Theory + demonstration
Session 5 – Bonus Content(Theory + demonstration)
It is recommended that you start this module early so that you can complete it within a week’s time. Please also try writing/running the codes with the experts. The presentation used in these sessions will be provided in the corresponding ‘Session Summary’ segment. The lecture notes will be provided in the ‘Module Summary’ segment (In Session 4). Relevant resources for the demonstration will be provided in their respective segments.
Guidelines for In-Segment and Graded Questions
There will be a separate session for graded questions. The other sessions will contain non-graded questions. Each graded question in this module will have 10 marks for a correct response and 0 for an incorrect response. Each graded question will have only one attempt, while each non-graded question will have one or two attempts depending upon the type of question and the number of options.
People you will hear from in this module
Subject Matter Expert
Ajay Shukla
Senior Data Engineer
Ajay is currently working as a Senior Data Engineer with a leading ride-hailing service provider in the Middle East. He has been working for over eight years in the industry and is associated with top retail and entertainment organisations.
Report an error