In the previous segment, you learnt how dynamic tables are used for stream processing with SQL. In the upcoming video, you will take a look at some Table API operators that are available.
Let’s summarise what you learnt in this video.
The following functions are available to perform scan, projection and filter operations with the Table API.
- Select: It is similar to the SQL SELECT query. It performs the select operation, and (*) can be used to fetch all the columns of the table. It is available in both batch and stream processing.
- Alias: This function is used to rename the fields. It is available in both batch and stream processing.
- Where/Filter: It is similar to the SQL WHERE clause and is used to filter out values for which the expression returns a false value. It is available in both batch and stream processing.
Column operations
The following functions are available to perform operations on columns:
- AddColumns: This function is used to add new fields. It will throw an exception error if the field already exists in the table. It is available in both batch and stream processing.
- AddOrReplaceColumns: If the new field that is to be added already exists in the table, then the older version is replaced with the new field. It is available in both batch and stream processing.
- DropColumns: This operation is used to drop the existing fields. It is available in both batch and stream processing.
- RenameColumns: The names of the existing fields can be changed using this function. It is available in both batch and stream processing.
Aggregation operations
The following functions are available to perform aggregation operations:
- GroupBy Aggregation: It is similar to the SQL GROUP BY clause. It groups the rows based on the grouping keys and is available in both batch and stream processing.
- GroupBy Window Aggregation: It groups and aggregates a table on a group window and grouping keys. It is available in both batch and stream processing.
Joins
The following functions are available to perform join operations:
- Inner Join: It is similar to the SQL JOIN clause, that is, joins two tables. The tables that are to be joined should have distinct field names. It is available in both batch and stream processing.
- Outer Join: It is similar to the SQL LEFT / RIGHT / FULL OUTER JOIN clauses. It is available in both batch and stream processing.
Insert
The following functions are available to perform insert operations:
- Insert Into: It is used to perform insertion in the registered output table. It is available in both batch and stream processing.
Additional Reading
- Table API Operators – This page lists all the operators and functions available.
Report an error