Yes, you can certainly use up your CPU allocation on an M-10 database (at which point we offer online resizing as large as you want to go, all the way up to 192 CPUs and 1.5TiB RAM). Even still, I've been able to coax more than 10,000 IOPS from an M-10. (Actually, out of dozens of M-10s colocated on the same hardware all hammering away.)
You can get a lot more out of that CPU allocation with the fast I/O of a local NVMe drive than from the slow I/O of an EBS volume.
This will be faster than an equivalent RDS instance and will handle more of the operational lifecycle around failover and high-availability with less downtime than RDS.
Within PlanetScale's product lineup, Metal refers to the use of local NVMe drives. Nothing more. These extremely affordable sizes are indeed slices of larger boxen, though no resources are overcommitted.
We run on the same instance types the larger PlanetScale Metal sizes offer as whole instances. For Intel that's r6id, i4i, i7i, i3en, and i7ie. For ARM that's r8gd, i8g, and i8ge. (Right now, at least. AWS is always cookin' up new instance types.) Same story will soon be true for GCP.
We've engineered in protections from noisy neighbors in both CPU and I/O usage and we do not over-commit resources.
If your or another customer's workload grows and needs to size up we launch three whole new database servers of the appropriate size (whether that's more CPU+RAM, more storage, or both), restore the most recent backups there, catch up on replication, and then orchestrate changing the primary.
Downtime when you resize typically amounts to needing to reconnect i.e. it's negligible.
PlanetScale operates databases in AWS and GCP. There's no network latency penalty for choosing PlanetScale if you're hosting your app in one of those cloud providers (and in one of the many regions we operate in).
Just want to add that you don't necessarily need to invest in fancy disk-usage monitoring as we always display it in the app and we start emailing database owners at 60% full to make sure no one misses it.
RAID isn't the answer, either, for the record. In AWS and GCP, the CPU or RAM blowing up will cost you access to that local NVMe drive, too, no matter how much RAID you throw at it.
Yes, a single disk in a VPS or cloud provider has durability concerns. That's why EBS and products like it that pretend to be a single disk are actually several. Instead of relying on multiple block devices, though, we create that redundancy at a higher level by relying on multiple MySQL or Postgres servers for durability, each with a local NVMe drive for performance.
The question isn't how many orphaned rows do you have, it's whether it matters. Databases are wonderful but they cannot maintain every invariant and they cannot express a whole application. They're one tool in the belt.
I computed P(node failure within MTTR) = 0.00007 same as you. I extrapolated this to the outage scenario P(at least two node failures within MTTR) = P(node failure within MTTR)^2 * (1-P(node failure within MTTR)) + P(node failure within MTTR)^3 = 5.09 * 10^-9 which rounds to 0.0000001%.
A lagging replica, even one that just acknowledged a semi-sync write, will return stale results if you route a `SELECT` to it.
First and foremost, the extra copies of the data are for fault tolerance. In specific circumstances they may offer some slack capacity that you can use to serve (potentially stale) reads but they're never going to offer read-your-writes consistency.
The docs you quote are a bit obtuse but the "acknowledgement" is the row event being written to the binary log. "Fully executed and committed" is when it makes its way into InnoDB and becomes available for future reads.
I think we've got (1) covered elsewhere in the comment tree.
For (2), semi-synchronous replication is a MySQL term which we realize in Postgres is by using synchronous replication with ANY one of the available replicas acknowledging the write. This allows us to guarantee durability in two availability zones before acknowledging writes to clients.
In MySQL the _semi_ part of semi-synchronous replication refers to the write only needing to be written to the binary log on the replica and not (necessarily) applied to InnoDB. This is why a MySQL database might be both acknowledging semi-synchronous writes and reporting non-zero replication lag.
Good point, Max. I glossed over the "rather than" bit. We do, as you say, write to disks all over the place.
Even writing to one disk, though, isn't good enough. So we write to three and wait until two have acknowledged before we acknowledge that write to the client.
1. I don't know if there's a single name for this. I will point out that AWS EBS and Google Persistent Disk as industrial examples of distributed, replicated block devices are also providing durability via replication. They're just providing it at a lower level that ends up sacrificing performance. I'm struggling to come up with a citation but I think it's either Liskov or Lynch that offered a proof to the effect of achieving durability in a distributed system via replication.
2. The thinking laid out in the blog post you linked to is how we went about it. You can do the math with your own parameters by computing the probability of a second node failure within the time it takes to recover from a first node failure. These are independent failures, being on physically separate hardware in physically separate availability zones. It's only when they happen together that problems arise. The core is this: P(second node failure within MTTR for first node failure) = 1 - e^( -(MTTR node failure) / (MTBF for a node) )
3. This one's harder to test yourself. You can do all sorts of tests yourself (<https://rcrowley.org/2019/disasterpiece-theater.html>) and via AWS FIS but you kind of have to trust the cloud provider (or read their SOC 2 report) to learn how availability zones really work and really fail.
Fair to say 500GB is small, especially compared to some of the folks who've already migrated, but do note that it's 15x RAM on the benchmark machines, so we really were testing the whole database and not just the memory bandwidth of the CPUs.
You can get a lot more out of that CPU allocation with the fast I/O of a local NVMe drive than from the slow I/O of an EBS volume.