Now Kafka has ZK taken out and tiered storage coming in these factors are becoming mute points. With tiered storage Kafka has 2 layers and hence the benefits of broker/storage decoupling.
By comparison, Pulsar has broker, bookkeeper, S3, and ZK layers to contend with. This is why a second layer was never added to Kafka directly.
I'm lost too. Kafka auto creates topics by default. Maybe you're referring to being able to create more topics? But that seems to be unproven. Kafka's limit is metadata and Pulsar is more metadata dependent than Kafka.
> Pulsar will happily do with just 2x.
This is just wrong. Pulsar provides weaker guarantees than Kafka. It's a quorum based system. If you run with two replicas Pulsar can't provide F-1 guarantees which Kafka can.
1. is true, but if you want that data to move to a new node, it still needs to be replicated. Kafka's approach is to use tiered storage (which I believe is close to completion).
2. Kafka can read from a replica node. It's relatively new but it's there.
This smacks of being heavily one-product-focussed to me. Being a Kafka user it's hard enough managing and understanding one system, nevermind three or four joined together.
Maybe it's a bit faster or a bit more elastic, or whatever, who knows. What I really care about is whether I get called at 3am and in that regard the argument seems pretty weak. Kafka for all its woes is a solid system you know you can count on.
I'd much rather see someone come up with a truly innovative alternative that actually pushes the boundaries, rather than just copying what's there already, and adding a few window dressings.