In the last segment, you learn about consumers and producers. Let’s take a look at some of the challenges that can be faced and learn how to overcome them.
In the next video, let’s take a look at some of the challenges faced.
In the previous video, you learnt about some of the challenges that you may face when using Kafka. The most important one is to introduce parallelism while reading data from partitions. Also, we need to know which consumer is reading from which partition and which consumers belong to a particular application. Let’s learn how to overcome these issues using the concept of consumer groups.
In the previous video, you understood the concept of consumer groups. They are a set of consumer who are grouped together and are identified by a group ID. You also looked at different scenarios that might arise with respect to the number of consumers in a group and the number of partitions of a particular topic. You learnt that when the number of consumers in the group is equal to the number of partitions of the topic, it leads to optimal utilisation of resources. It is also the best case of parallelisation.
Kafka guarantees that one partition of a topic cannot be associated with more than one consumer of the same group at the same time.
Additional Reading
Consumer groups: You can refer to this link if you want to read more on how Kafka brings the best of queueing and pub-sub using consumer groups.
Report an error