In the previous segment, you learnt about the basic concepts of the Table API/SQL. In the following video, our expert will explain the steaming concepts.
Let’s summarise your learnings from this video.
Streaming concepts
Table API and SQL of Apache Flink are two unified APIs for batch and stream processing. This means that the Table API and SQL queries have the same semantics regardless of the type of input, which can be either bounded batch data or unbounded stream data. SQL queries were designed for batch processing, and so, queries on unbounded streams are not well-understood. Flink can achieve the semantics on unbounded data using SQL queries. For this, it uses a dynamic table.
Dynamic tables
Dynamic tables are the core of the Table API and SQL support for stream processing. Static tables represent batch data, whereas dynamic tables change over time.
Dynamic tables can also be queried similar to static tables. A dynamic table query yields a continuous query, which is never ending and produces a dynamic table as a result. The result dynamic table is continuously updated to reflect the changes in the input dynamic table. The result of a continuous query is semantically equivalent to the query being executed on the static tables in the batch mode.
Stream processing with SQL is depicted in the image given below.
- An unbounded stream of data is converted to a dynamic table.
- A continuous query is executed on an input dynamic table, which produces another dynamic table as the output.
- Finally, the output dynamic table is converted back to a datastream
Report an error