In this segment, you will learn about the Hive operator.
We recommend that you follow along with the demonstrations in your own EMR instance.
Note:
As mentioned previously, make sure that you have the current enabled JDK as JDK 8 for using the Hive operator.
In the upcoming video, Ajay will introduce you to the Hive operator.
So, in the video, you learnt the theory related to the Hive operator in Airflow.
The HiveOperator is used to connect to Hive using hive_conn_id and execute Hive queries.
Some of the important parameters/arguments for the HiveOperator are listed below:
- Hql: Hive query to be executed
- hive_cli_conn_id: Hive connection
Note:
The task_id and dag arguments have to be mentioned for all operators.
In the next video, we will start with the actual demonstration of the Hive operator.
You can find the code and other resources used in the demonstration attached below:
The document provided below details the steps followed in the demonstration.
In the next segment, you will learn about the Spark operator in Airflow.
Additional Reading
You can visit the following link for the source code for the HiveOperator: HiveOperator.