HackerTrans
TopNewTrendsCommentsPastAskShowJobs

mfreed

no profile record

Submissions

Replacing EBS and Rethinking Postgres Storage from First Principles

tigerdata.com
105 points·by mfreed·il y a 9 mois·59 comments

comments

mfreed
·il y a 4 mois·discuss
github.com/timescale/tigerfs
mfreed
·il y a 4 mois·discuss
Hey, author here. Just noticed this.

Yes, TigerFS works well with local PG - in fact, all of my testing framework uses this setup.

Big upside of this is latency; main downside is that such a deployment is primarily "single user mode."

One very interesting approach of this is ability to transparently share state / collaborate across machines, with these transactionally serialized at the common database.

I primarily work with Postgres, including both at scale and in very ephemeral environments. So just haven't looked at all what doing with this SQLite would mean.
mfreed
·il y a 8 mois·discuss
None of this. It's in the blog post in a lot of detail =)

The 5ms write latency is because the backend distributed block storage layer is doing synchronous replication to multiple servers for high availability and durability before ack'ing a write. (And this path has not yet been super-performance-optimized for latency, to be honest.)
mfreed
·il y a 8 mois·discuss
Currently support multi-AZ clusters and multi-region disaster recovery (continuous PITR between regions).

We're continuing to evaluate demand for multi-region clusters, love to hear from you.
mfreed
·il y a 8 mois·discuss
I'm not aware of any published source for this time limit, nor ways to reduce it.

The docs do say, however, "If the volume has been impaired for more than 20 minutes, you can contact the AWS Support Center." [0] which suggests its some expected cleanup/remount interval.

That is, it is something that we regularly encounter when EC2 instances fail, so we were sharing from personal experience.

[0] https://docs.aws.amazon.com/ebs/latest/userguide/work_volume...
mfreed
·il y a 8 mois·discuss
Tiger Cloud certainly continues to run on AWS. We have built it to rely on fairly low-level AWS primitives like EC2, EBS, and S3 (as opposed to some of the higher-level service offerings).

Our existing Postgres fleet, which uses EBS for storage, still serves thousands of customers today; nothing has changed there.

What’s new is Fluid Storage, our disaggregated storage layer that currently powers the new free tier (while in beta). In this architecture, the compute nodes running Postgres still access block storage over the network. But instead of that being AWS EBS, it’s our own distributed storage system.

From a hardware standpoint, the servers that make up the Fluid Storage layer are standard EC2 instances with fast local disks.
mfreed
·il y a 8 mois·discuss
A few datapoints that might help frame this:

- EBS typically operates in the millisecond range. AWS' own documentation suggests "several milliseconds"; our own experience with EBS is 1-2 ms. Reads/writes to local disk alone are certainly faster, but it's more meaningful to compare this against other forms of network-attached storage.

- If durability matters, async replication isn't really the right baseline for local disk setups. Most production deployments of Postgres/databases rely on synchronous replication -- or "semi-sync," which still waits for at least one or a subset of acknowledgments before committing -- which in the cloud lands you in the single-digit millisecond range for writes again.