InterviewSolution
| 1. |
what is the consumer group in Kafka? |
|
Answer» Let's first understand the concept of the consumer in Kafka architecture. The consumers are the system or process which subscribe to topics created at the Kafka BROKER. The producer's system sends messages to topics and once messages are committed successfully then only subscribers systems are allowed to read the messages. The consumer group is tagging of consumers system in such a way to make it multi-threaded or multi-machine system. As we can see in the above diagram, two consumers 1 & 2 are being tagged in the same group. Also, we can see that individual customers reading DATA from different partition of topics. Some common characteristic of consumer groups are as follows:
The recommendation for the consumer group suggests having a similar number of consumer instances in line with several partitions. In case if we will go with a greater number of consumers then it will result in excess customers sitting idle resulting in wasting resources. In the case of partitions numbers greater then it will result in the same consumers reading from more than one partition. This should not be an issue until the time ordering of messages is not IMPORTANT for the use case. Kafka does not have inbuilt SUPPORT for the ordering of messages across different partitions. This is the reason why Kafka recommends to have the same number of consumers in line with partitions to maintain the ordering of messages. |
|