Architecture decisions in Neon, a serverless Postgres service(neon.tech)
neon.tech
Architecture decisions in Neon, a serverless Postgres service
https://neon.tech/blog/architecture-decisions-in-neon
16 comments
(neon ceo)
Yes you will have an S3 fetch latency for downloading a layer file for such data. However we don't page out aggressively so it's a very unlikely event. Most of the time you will be fetching from a locally attached SSD which is superior and more predictable than EBS.
Where it works super well is the long tail of small and rarely used database - it pushes our costs way down.
Another advantage is that if the whole region goes down the S3 copy survives.
Finally if you separate storage and compute you can be serverless and scale to 0.
Where it works super well is the long tail of small and rarely used database - it pushes our costs way down.
Another advantage is that if the whole region goes down the S3 copy survives.
Finally if you separate storage and compute you can be serverless and scale to 0.
Thanks Nikita!
I'm more curious how someone is allowed to have the time and money to build this?
I would love to spend 1-3 years with other good people to just build good tech :(
I would love to spend 1-3 years with other good people to just build good tech :(
Business as usual: somebody funds it--not necessarily the developers--in exchange for a slice of the pie if the project is successful.
If it will be succesfull, time will tell. But serverless postgres would be really sweet to have. Hell, AWS itself should be funding this project. I run my Postgres for hobby projects at home because AWS Aurora serverless postgres is very expensive and doesn't scale down to zero. I wouldn't mind paying a dollar or two for storage the months I don't use that project, and a little more when I actually need to query my databases. At work, my $$$$$$ employer pays $$-employees to poke at $$$-devs who forget to shut down an Aurora serverless Postgres.
If it will be succesfull, time will tell. But serverless postgres would be really sweet to have. Hell, AWS itself should be funding this project. I run my Postgres for hobby projects at home because AWS Aurora serverless postgres is very expensive and doesn't scale down to zero. I wouldn't mind paying a dollar or two for storage the months I don't use that project, and a little more when I actually need to query my databases. At work, my $$$$$$ employer pays $$-employees to poke at $$$-devs who forget to shut down an Aurora serverless Postgres.
This is a very deliberate effort which is venture funded. It started with a deck three years ago and a team of myself, Heikki and Stas.
Since it’s my second company (prev was singlestore) the early funding was relatively easy. Later funding was done based on our traction.
Since it’s my second company (prev was singlestore) the early funding was relatively easy. Later funding was done based on our traction.
Uj thanks for that insight.
GZ :)
GZ :)
I'm pretty bullish on this tech.
A clean, low-friction separation of storage and compute for a top-tier database like Postgres is huge. I think it will take some time for the implications to sink in though, because it changes some assumptions about how to integrate a databases with a system that are so fundamental that we rarely think about it... (not to mention, it probably doesn't make sense to change existing systems, where the fundamental system decisions have already been made.)
We tend to centralize the database in system design because they are complex and expensive to run, and you can amortize those costs the more you put into a single database. But of course, that also makes the central database bigger, more complex and more expensive. And once everything is in a central store, we tend to bind everything together (with joins, references, procs, etc. Each step in that direction tends to make sense on its own, but the end result is the "big ball of mud" architecture where everything is interconnected and there's no separation of concerns, which makes the system fragile and increasingly difficult to change.
With this tech the pressure to centralize the database is relieved.
E.g., if you have a situation where you are adding some new, logically distinct data to your system -- maybe just a few tables and maybe with a variable load, especially at first -- traditionally you probably would lean hard toward adding this to an existing central database since the overhead of spinning up and running a new one would be prohibitive. But now you can just make that a separate database -- spinning up a managed one is easy and you only need to pay for as much storage and compute as you actually need to use (which is really about the same as if you had put the tables in an existing central database anyway).
Of course, systems would still tend to have significant central data, but that could be based on actual app concerns, not operational ones.
Anyway, I'm loving this and can't wait to see the long term impact.
You can have as many databases as logically makes senseseparate data logically without exploding the operational cost of your system. E.g. you can spin out a separate a database that has light (or even no load) most of the time yet can seamlessly scale to heavy load. And it can do this independent of how much data is in it, and you don't need to worry about it's affect on day-to-day data access in your system since you
A clean, low-friction separation of storage and compute for a top-tier database like Postgres is huge. I think it will take some time for the implications to sink in though, because it changes some assumptions about how to integrate a databases with a system that are so fundamental that we rarely think about it... (not to mention, it probably doesn't make sense to change existing systems, where the fundamental system decisions have already been made.)
We tend to centralize the database in system design because they are complex and expensive to run, and you can amortize those costs the more you put into a single database. But of course, that also makes the central database bigger, more complex and more expensive. And once everything is in a central store, we tend to bind everything together (with joins, references, procs, etc. Each step in that direction tends to make sense on its own, but the end result is the "big ball of mud" architecture where everything is interconnected and there's no separation of concerns, which makes the system fragile and increasingly difficult to change.
With this tech the pressure to centralize the database is relieved.
E.g., if you have a situation where you are adding some new, logically distinct data to your system -- maybe just a few tables and maybe with a variable load, especially at first -- traditionally you probably would lean hard toward adding this to an existing central database since the overhead of spinning up and running a new one would be prohibitive. But now you can just make that a separate database -- spinning up a managed one is easy and you only need to pay for as much storage and compute as you actually need to use (which is really about the same as if you had put the tables in an existing central database anyway).
Of course, systems would still tend to have significant central data, but that could be based on actual app concerns, not operational ones.
Anyway, I'm loving this and can't wait to see the long term impact.
You can have as many databases as logically makes senseseparate data logically without exploding the operational cost of your system. E.g. you can spin out a separate a database that has light (or even no load) most of the time yet can seamlessly scale to heavy load. And it can do this independent of how much data is in it, and you don't need to worry about it's affect on day-to-day data access in your system since you
We already have it with YugabyteDB, which is open source (unlike Neon), but doesn't seem to be the success story it should have been. At least I've heard barely a peep about it. Maybe they're not so great at marketing.
Yugabyte is architected similarly to other similar databases, with an SQL layer on top of a distributed, replicated key/value store based on RocksDB. The different being that the SQL layer is Postgres.
Yugabyte tightly couples the two, though, though it seems like this is mostly for convenience, in that the "TServer" (tablet server) is managed by Yugabyte itself. I wonder if it could support an architecture more similar to TiDB, where the storage is independently scalable from compute.
CockroachDB and TiDB are based on a similar design, but Yugabyte stands out as being a "real" Postgres. It even supports normal Postgres extensions via "CREATE EXTENSION".
Yugabyte is architected similarly to other similar databases, with an SQL layer on top of a distributed, replicated key/value store based on RocksDB. The different being that the SQL layer is Postgres.
Yugabyte tightly couples the two, though, though it seems like this is mostly for convenience, in that the "TServer" (tablet server) is managed by Yugabyte itself. I wonder if it could support an architecture more similar to TiDB, where the storage is independently scalable from compute.
CockroachDB and TiDB are based on a similar design, but Yugabyte stands out as being a "real" Postgres. It even supports normal Postgres extensions via "CREATE EXTENSION".
I mostly agree with the commenter you’re replying to - but I think you bring up a good point. Once you start disconnecting tables from one another, and end up with lots of small data stores, does it still make sense to bother with the Postgres at all?
I think that its value as a layer becomes unclear. Sure its battle-tested and has great query planning and similar tools, but those base based on assumptions that are largely violated with the re-architecture.
I think the architectural decoupling benefits mentioned are basically already proven though. I’ve worked at places that had many NoSQL tables per team, each logically distinct, and at places that had one database with 50+ tables in it. The breakup can be nice, in the same way that “serverless” is nice - data storage becomes entirely abstracted to the developer.
I think that its value as a layer becomes unclear. Sure its battle-tested and has great query planning and similar tools, but those base based on assumptions that are largely violated with the re-architecture.
I think the architectural decoupling benefits mentioned are basically already proven though. I’ve worked at places that had many NoSQL tables per team, each logically distinct, and at places that had one database with 50+ tables in it. The breakup can be nice, in the same way that “serverless” is nice - data storage becomes entirely abstracted to the developer.
The problem that shared nothing systems lose compatibility with the single node system. So planetscale and yugabyte are not 100% compatible with Postgres. They are also very expensive at the low end hence the free tiers are discontinued.
I have a thread about it on twitter: https://x.com/nikitabase/status/1723579854266966399?s=46
I have a thread about it on twitter: https://x.com/nikitabase/status/1723579854266966399?s=46
Neon is open source and under a good license: Apache 2
One thing that is very important is what you charge for. Software or service. We are only charging for the cloud service and it has major implications in the “company architecture”. When you charge for software like yuga and roach and for the cloud you end up with two companies in one.
Counterexample is snowflake - they got it right !
Counterexample is snowflake - they got it right !
Did I miss something? I always thought that Neon is open source.
For some reason I thought it wasn't open source. Looks like I'm wrong!
This is pretty much the first part of the strategy: database is just a url.
We have more cool stuff coming!
We have more cool stuff coming!
The article mentions that a consequence of separating storage from compute is that compute nodes cache hot pages in memory and load cold pages from object storage (like S3?) when needed. Does anyone know what are the consequences of this decision. In case of a query that touches multiple rarely used pages, would that incur high latency and ingress? How does that penalty compare to a vanilla postgres running on AWS and storing pages on EBS?