The impact is pretty clear. Below we load ten events using a given key. We add a partition, then load ten more events. Notice the partition differs after adding a partition, even though the key is the same.
Category: Kafka
Python KafkaProducer and lost events
While testing the python kafka package and its associated KafkaProducer class, I found that events were “lost”; i.e., they were never successfully persisted in the destination topic. Various blogs suggest setting a sleep(2) call, or whatever, which seemed odd. I…