I think we have a lot of DBs specialised for heavy write, like anything with LSM-tree in their base.
My vision is that there will be more movement in this direction, but still, we need first to understand limits of "easy to work with" databases like Postgres. It's easy to underestimate what Postgres can do.
And only when it's not enough, move to something LSM-tree-based.
Small badges will be helpful, though. I added hovers in some places, like for latency numbers, but I think more visible badges will work better, thanks!
Good points, thanks. On huge pages: this is also about RDS vs self-managed EC2 Postgres. RDS effectively has "on" by default, but default self-managed (that I benchmarked) is "try" which is effectively "off". I'll update the methodology page to cover that, and, yeah, it makes sense to cover that separately.
io2 is on my future-work list. And agree, I have the same feelings about IOPS.
Yes! This was my initial dilemma - whether to test RDS or self-hosted Postgres on EC2. I decided to start with EC2 to be a bit more "pure", and remove cost overhead of RDS.
But support for RDS is my next candidate for development. Plus, comparison would also be interesting.
Good point! I kept the configuration of the Postgres pretty close to the defaults, and it would be interesting to compare it with the same default Aurora Postgres.
And it should be easy to add - I'll check it, thanks!
I got frustrated seeing how system design questions differ from the real world, and how big tech companies foster a scalability-first mindset across the industry, while cost-efficiency and critical thinking about real product requirements are much more important.
I've checked Bitly public numbers and typical URL shortener interview question requirements. And I've benchmarked a simple implementation as much as possible across different hardware. My obscure NAS already handles 2540 writes/sec. The PostgreSQL setup on AWS does 15900. Both have 3 billion pre-filled rows. While the interview asks for 1160, the real systems have only 93 writes/s.
I got frustrated seeing how system design questions differ from the real world, and how big tech companies foster a scalability-first mindset across the industry, while cost-efficiency and critical thinking about real product requirements are much more important.
I've checked Bitly public numbers and typical URL shortener interview question requirements. And I've benchmarked a simple implementation as much as possible across different hardware. My obscure NAS already handles 2540 writes/sec. The PostgreSQL setup on AWS does 15900. Both have 3 billion pre-filled rows. While the interview asks for 1160, the real systems have only 93 writes/s.
Especially, cost-efficiency. I'll think how to put it, thanks :)