Imantouch

Engineering

Sizing a Kafka cluster you will not regret

Published on 15 July 2026 · By Imantouch · 6 min read

Most Kafka sizing exercises start from throughput. Retention, partition count and the recovery window decide more, and they are the numbers people skip.

Throughput is the number everyone asks for first, and it is rarely the constraint. A three-broker cluster on ordinary hardware absorbs more than most organisations produce. What actually sizes the cluster is how long you keep the data, how many partitions you end up with, and how quickly you need to be back after losing a broker.

Retention is a storage decision

Seven days of retention at 200 MB/s is roughly 120 TB before replication, and three times that after. Teams routinely set retention "just in case" and discover the bill later. Decide retention per topic, not per cluster.

Partition count is a rebalance decision

Partitions are cheap until they are not. Every partition is an open file handle, a replication stream, and a unit of rebalancing work. A cluster with 200,000 partitions does not fail on throughput; it fails on the twenty minutes it takes to elect leaders after a rolling restart.

The recovery window is the real requirement

Ask what happens when a broker dies during business hours. If the answer is "we need to be back in ten minutes", the cluster is sized by replication catch-up speed, not by peak write rate. That one question changes the disk type, the instance size and the number of brokers, more than any throughput figure will.

What we do first

Before proposing a topology, we measure a week of real traffic and pull three numbers: peak sustained write rate, retention actually needed per topic, and the acceptable recovery window. Everything else follows from those.

Let’s talk about what your platform needs

Tell us what you are building, or what is keeping you up at night. We will tell you plainly whether we are the right team for it.

Sizing a Kafka cluster you will not regret | Imantouch