Show HN: StreamHouse – S3-native Kafka alternative written in Rust(github.com)
github.com
Show HN: StreamHouse – S3-native Kafka alternative written in Rust
https://github.com/gbram1/streamhouse
8 comments
Running the quickstart snippet I get a bunch of errors and a warning that my anthropic key is not set. Smells vibe coded!
where are you using this in production?
For systems that can tolerate slightly higher latency, and prefer operational simplicity.
A good example would be a ecommerce platform that is streaming order history into a data warehouse. They likely don’t need sub-millisecond latency that Kafka provides, and can use a simpler, significantly cheaper option to achieve the same result
I thought we were all ditching minio as it is no longer maintained
I still find it useful as a local blob storage and no big code changes to cut over to s3 since it’s the same APIs
Try rustfs
I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost.
How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box.
What's there today: - Producer API with batching, LZ4 compression, and offset tracking (62K records/sec) - Consumer API with consumer groups, auto-commit, and multi-partition fanout (30K+ records/sec) - Kafka-compatible protocol (works with existing Kafka clients) - REST API, gRPC API, CLI, and a web UI - Docker Compose setup for trying it locally in 5 minutes
The cost model is what motivated this. Kafka's storage costs scale with replication factor × retention × volume. With S3 at $0.023/GB/month, storing a TB of events costs ~$23/month instead of hundreds on broker EBS volumes.
Written in Rust, ~50K lines across 15 crates. Apache 2.0 licensed.
GitHub: https://github.com/gbram1/streamhouse
Happy to answer questions about the architecture, tradeoffs, or what I learned building this.