Let’s summarise your learnings from this session.
So, here is a summary of what you learnt in this session:
- You started with the structured streaming API and learnt about its advantages over the dstream API and learnt about its advantages over the dstream API.
- Next, you learnt about the differences between static and streaming DataFrames and set up a simple application to consume streaming data.
- Next, you learnt about triggers and also learnt about the various output modes with a coding demo.
- This was followed by a few transformations and their uses as demonstrated through codes, and we concluded by learning about joins and their types.
In the next session, you will learn about some of the advanced features of Spark Structured Streaming such as windowing and watermarking.
The PPT that was used throughout this session is attached below.