Kafka components one by one
Courtesy : Stéphane Maarek and Learning Journal Installation mkdir ~/Downloads curl " https://www.apache.org/dist/kafka/2.1.1/kafka_2.11-2.1.1.tgz " -o ~/Downloads/kafka.tgz mkdir ~/kafka && cd ~/kafka tar -xvzf ~/Downloads/kafka.tgz --strip 1 Partition: A Topic can be broken in to multiple partition, so that each partition can reside in a separate Broker instance, thus achieving parallelism for that specific topic (KPI). New message written to a topic: Kafka: Decides which partition will the message go, kind of load balancing, Que: How does it knows, which partition is free ? or it's offset is not full(12) ? Even of you parallel produce message in 2 partition, the bottleneck can still be in consumer side? 4. Topic Partition replication and leader 5. Load distribution and acknowledgment Producer just send data to " BrokerHost:TopicName" Kafka: Will decide, which broker instance and which partition of that T