Producers and Consumers

In the previous segment, you learnt about topics and partitions. In this segment, let’s learn more about producers and consumers which are the main components of Kafka Internals.

Let’s start with producers in the next video.

In the previous video, you understood the concept of producers. Producers are responsible for writing messages to any Kafka topic. Any message contains a key and a value. Attaching a key to messages will ensure that messages with the same key always go to the same partition in a topic. When you do not provide a key to your messages, the round-robin distribution is done to ensure that no partition gets overloaded with all the messages.

In the next video, you will learn about consumers.

In the previous video, you learnt about consumers. They subscribe to topics and read data from these subscribed topics. They read data in a sequential manner. The message that is written first is the first one to be consumed, and the one that is written last is the last one to be consumed. When a single consumer is trying to read messages from more than one partition, the ordering is guaranteed at the partition level only and not across the partitions as seen with the help of animation.

You will see a practical implementation of producers and consumers in the next session.

Additional Reading

Message Keys: Read more about what happens when you set the key or when you do not specify a key for a message.

Report an error