Now that you have understood how Kafka Connect works, in this segment, you will learn about Kafka Streams. Let’s watch the next video to learn about this in detail.
Kafka Streams is a client library used for building applications and microservices. All the input and output data are stored in the Kafka cluster itself. Kafka Streams is one of the APIs of Apache Kafka. you can use it to build scalable and fault-tolerant applications.
In the next video, you will understand the important of Kafka Streams with the help of a scenario.
In the video above, you saw a scenario that involved filtering mobile and desktop data and storing the data into two different Kafka topics. One of the approaches to do this was using producers. In this case, the consumer would read the full data, divide it into mobile data and desktop data and then again write it into two different topics. This approach might lead to low latency at each step but ultimately result in a significantly high latency, which is not the desired outcome. This is where Kafka streams plays an important role.
Using the Kafka Streams application, you can directly read the data from one topic, filter the different types of data and then write the data back to different topics. In other words, in Kafka streams, we read from one topic, process the data and then write it again to some other topic.
In the next video, you will understand the architecture of Kafka Streams and go through some scenarios using Kafka Streams.
In the video above, you saw the architecture of Kafka Streams. Kafka streams reads data from one Kafka topic, applies some operations to this data and then writes it back to some other Kafka topic. It reads and writes only from and to Kafka topics. It is used for simple transformations, processing of data, some sort of anomaly detection and in monitoring the health of different systems.
Additional Reading
Microservices- You can read about microservices from this link. It lists characteristics and examples of microservices.
Kafka Streams- Read from here to enhance your basic understanding of Kafka Streams.
Kafka Streams- Official Documentation of Kafka Streams lists some of its key features and its use cases.
Report an error