Managed CockroachDB: Geo-Distributed Database as a Service(cockroachlabs.com)
cockroachlabs.com
Managed CockroachDB: Geo-Distributed Database as a Service
https://www.cockroachlabs.com/blog/launching-managed-cockroachdb/
50 comments
This is equal parts impressive and scary. It's awesome from a stability perspective, but I'm not completely comfortable with "dial home" functionality apparently being enabled by default in a database system.
It's printed "in bold letters on the label";
and like any sane db deployment you should firewall it off anyway.
and like any sane db deployment you should firewall it off anyway.
If i remember it right, you can switch off the telemetry during installation
Here's the details: https://www.cockroachlabs.com/docs/stable/diagnostics-report...
That is great but I would rather have software which does not crash than a team of very enthusiastic developers who are eager to fix bugs. One example of this would be FoundationDB or Riak. I never managed to crash Riak in a production environment, I haven't used FoundationDB in production to say the same but based on the method they use to build the software is just mind blowing and afaik verrry efficient.
Well obviously, which is what they're striving to achieve by putting measures like this in place
So what's the price? For Spanner, which is perhaps their closest competitor, it takes a few seconds to find at least the list price, before discounts and such.
Databases are the core of a digital business. The opportunity cost of losing data is huge. Customers won't blindly trust a product with their data until they understand the pricing. Nobody is putting this DB into production on a 30-day free trial with no pricing info.
CockroachDB focuses a lot on showing technical users that they can trust the product. If they want to make money, then they need to become a trustworthy vendor, too.
CockroachDB focuses a lot on showing technical users that they can trust the product. If they want to make money, then they need to become a trustworthy vendor, too.
They sure don't make it easy to find on their site, though this post breaks down some of it: https://www.cockroachlabs.com/blog/aurora-price-vs-cockroach...
Cockroaches team also have their customers as a top level nav item now, though Baidu is the only one I recognize: https://www.cockroachlabs.com/customers/
Cockroaches team also have their customers as a top level nav item now, though Baidu is the only one I recognize: https://www.cockroachlabs.com/customers/
They compared the free version pricing, not enterprise.
They want $10k per year per core, making it pretty much the most expensive thing you could possibly use as a database and multiple times more expensive than SQL Server over the course of a few years, and far more expensive than even using Oracle Cloud. It's a really well built product, but the pricing is not even close to compelling.
Cockroach Labs employee here.
CockroachDB has pricing that is tiered to different customer needs and sizes. Reach out with a business need and you will see that our pricing is incredibly competitive (certainly lower than Oracle or SQL Server) and delivers a cloud-native, geo-distributed SQL solution that is unmatched.
CockroachDB has pricing that is tiered to different customer needs and sizes. Reach out with a business need and you will see that our pricing is incredibly competitive (certainly lower than Oracle or SQL Server) and delivers a cloud-native, geo-distributed SQL solution that is unmatched.
CockroachDB is looking more and more like a viable replacement for MySQL (now that they have importing mysqldump in beta)
However key management in clusters is an absolute nightmare. They need to look at Docker Swarm for how to build a usable "join" mechanism. Until then, the majority of folks just use --insecure in production.
However key management in clusters is an absolute nightmare. They need to look at Docker Swarm for how to build a usable "join" mechanism. Until then, the majority of folks just use --insecure in production.
CockroachDB engineer here.
> However key management in clusters is an absolute nightmare. They need to look at Docker Swarm for how to build a usable "join" mechanism.
I'll contend that key management is one of the great unsolved problems in software deployment.
I don't want to diminish the value of removing friction from the user experience, but fundamentally there is not that much of a difference between providing a token directly as a command line argument vs. providing a certificate that lives on disk. I'm curious if your frustration is mostly from the overhead of creating node certificates or if there's a deeper problem I'm not seeing.
I should also mention: this is an area we're actively looking to improve in! If you have time to write up a slightly more detailed proposal as a GitHub issue, we'd be happy to entertain it.
> However key management in clusters is an absolute nightmare. They need to look at Docker Swarm for how to build a usable "join" mechanism.
I'll contend that key management is one of the great unsolved problems in software deployment.
I don't want to diminish the value of removing friction from the user experience, but fundamentally there is not that much of a difference between providing a token directly as a command line argument vs. providing a certificate that lives on disk. I'm curious if your frustration is mostly from the overhead of creating node certificates or if there's a deeper problem I'm not seeing.
I should also mention: this is an area we're actively looking to improve in! If you have time to write up a slightly more detailed proposal as a GitHub issue, we'd be happy to entertain it.
I encounter way too much server software that seems written by people that are insensitive to how modern deployments work.
Any instructions that involve "just login here, just run X, just fiddle with that file over there", "just chant these incantations and perform our ritual startup dance",etc are kind of broken. The more convoluted those instructions get, the harder and more error prone it gets to automate the deployment in a sane way. Manually messing with files on a filesystem happens on development machines and laptops, never on production servers (unless you are dealing with amateurs). And even there I would argue running docker is the preferred way these days.
For me software comes in two forms these days. If it is stateful, I might not run it in docker and I'll be building an AMI using packer and ansible. If it is not, it is a docker image. In both cases I build one image or container (on a build server) for one thing that gets booted many times on many servers in the context of some deployment template. All of my configuration management happens at image build time (packer or docker). After that no modifications should be needed to their file systems and they should be read only for all practical purposes. Any data goes into separate volumes.
At boot time you inject all relevant configuration via environment variables. Ideally that is the same for all nodes of a type with no variation. Ideally the software understands that environment variables are (by far) the preferred mechanism to take configuration. Surprisingly few packages seem to get this right and assume some grey beard is going to ssh into a server and be all over their vintage nineteen seventies configuration files with extra weird syntax in some completely non standard location. It's been decades since software deployment actually worked that way. These days those files get templated, written once and never modified again. The template cruft gets driven from a script and that script is driven by environment variables. The more of that cruft is needed, the harder it gets to deploy stuff.
So, insisting on per node provisioning of certificates to some location in a filesystem kind of really sucks. I can totally see devops people disabling that instead and relying on firewalls, security groups and network security instead.
Any instructions that involve "just login here, just run X, just fiddle with that file over there", "just chant these incantations and perform our ritual startup dance",etc are kind of broken. The more convoluted those instructions get, the harder and more error prone it gets to automate the deployment in a sane way. Manually messing with files on a filesystem happens on development machines and laptops, never on production servers (unless you are dealing with amateurs). And even there I would argue running docker is the preferred way these days.
For me software comes in two forms these days. If it is stateful, I might not run it in docker and I'll be building an AMI using packer and ansible. If it is not, it is a docker image. In both cases I build one image or container (on a build server) for one thing that gets booted many times on many servers in the context of some deployment template. All of my configuration management happens at image build time (packer or docker). After that no modifications should be needed to their file systems and they should be read only for all practical purposes. Any data goes into separate volumes.
At boot time you inject all relevant configuration via environment variables. Ideally that is the same for all nodes of a type with no variation. Ideally the software understands that environment variables are (by far) the preferred mechanism to take configuration. Surprisingly few packages seem to get this right and assume some grey beard is going to ssh into a server and be all over their vintage nineteen seventies configuration files with extra weird syntax in some completely non standard location. It's been decades since software deployment actually worked that way. These days those files get templated, written once and never modified again. The template cruft gets driven from a script and that script is driven by environment variables. The more of that cruft is needed, the harder it gets to deploy stuff.
So, insisting on per node provisioning of certificates to some location in a filesystem kind of really sucks. I can totally see devops people disabling that instead and relying on firewalls, security groups and network security instead.
It's not bad at all when you use something like Kubernetes, which they recommend, since cert provisioning is built into Kubernetes.
Huh? A modern infrastructure setup should have PKI in place to issue certificates to nodes. You are making it sound like SSL has no place in the modern world.
Not making that point at all. Just making the point that related certificates and keys need to be looked up at run time from some secure place rather than be provisioned onto a file system at build time or via some hard to automate sysadmin ritualistic dance on the command line post startup. If in amazon, the secure parameter store is a good place for storing secrets. They also provision nice EC 2 credentials to hosts allowing you to access resources securely without having to rely on silly ssh keys baked into images or passwords passed around insecurely. And of course they allow you to centrally manage ssl certificates as well and manage provisioning those to things like load balancers. Kubernetes has nice infrastructure for this as well.
The point is that this stuff should never be baked into docker or ec2 images and having software expecting to find these things in some file on the filesystem actively makes it harder to configure it in a sane way.
I've seen quite a few docker images that require me to mount a volume with certificate files to work at all. The only way to beat some sense into that is to customize the Docker file and template these files (e.g. dockerize is great for this). Sane deployment automation seems to be an afterthought with many developers.
Since you brought up SSL, a nice innovation a few years ago was https://letsencrypt.org/ which uses short lived certificates that need to be renewed regularly. That sounds like a good practice to do anywhere you use any kind of ssl connections. That would require automation to refresh those certificates of course. Not just once on startup but also before they expire. However, if the software reads keys only once at startup from a file, part of that scripting is also becomes orchestrating a restart; which I imagine can get a bit hairy in the case of a clustered database product.
The point is that this stuff should never be baked into docker or ec2 images and having software expecting to find these things in some file on the filesystem actively makes it harder to configure it in a sane way.
I've seen quite a few docker images that require me to mount a volume with certificate files to work at all. The only way to beat some sense into that is to customize the Docker file and template these files (e.g. dockerize is great for this). Sane deployment automation seems to be an afterthought with many developers.
Since you brought up SSL, a nice innovation a few years ago was https://letsencrypt.org/ which uses short lived certificates that need to be renewed regularly. That sounds like a good practice to do anywhere you use any kind of ssl connections. That would require automation to refresh those certificates of course. Not just once on startup but also before they expire. However, if the software reads keys only once at startup from a file, part of that scripting is also becomes orchestrating a restart; which I imagine can get a bit hairy in the case of a clustered database product.
> I'll contend that key management is one of the great unsolved problems in software deployment.
Docker Swarm and Salt minions are really good examples of how it should work from a user interaction standpoint. Let me spin up a dozen or a hundred instances from the same configuration management template without having to seed each one with a secret, or build my own tooling to distribute certs to them. From one central point I approve new joins. Manage the CA and certificates within the cluster, don't expect your customers to do it.
Make a big poster for your office that says "Our real customers don't know how to do devops." MySQL took off because dumb people like me can get it to work.
Thanks for listening.
Docker Swarm and Salt minions are really good examples of how it should work from a user interaction standpoint. Let me spin up a dozen or a hundred instances from the same configuration management template without having to seed each one with a secret, or build my own tooling to distribute certs to them. From one central point I approve new joins. Manage the CA and certificates within the cluster, don't expect your customers to do it.
Make a big poster for your office that says "Our real customers don't know how to do devops." MySQL took off because dumb people like me can get it to work.
Thanks for listening.
Any chance of integration with k8s/istio cert management? A quick Google search doesn't bring up anything relevant.
>fundamentally there is not that much of a difference between providing a token directly as a command line argument vs. providing a certificate that lives on disk.
certificates with related infrastructure is more like a well-managed injected security "aspect" whereis key on the command line is an architecture style of a kitchen sink ad-hoc mix-in.
certificates with related infrastructure is more like a well-managed injected security "aspect" whereis key on the command line is an architecture style of a kitchen sink ad-hoc mix-in.
Have you looked at Vitess? Slack and Youtube use it.
How is Cloud Spanner adoption nowadays? I seem to be hearing lately that more and more teams are looking seriously into adopting it.
They just added SQL DML (insert/update/delete) a couple of weeks ago. Adoption will probably head up now. https://cloud.google.com/blog/products/databases/develop-and...
Edited to note "SQL DML"...
Edited to note "SQL DML"...
To clarify, you could always do these things via the native RPC interface. They added these things to the SQL interface.
Spanner is absolutely amazing. If you can live with the fact that you’re 100% vendor locked to google.
I can’t, so I run Postgres. But cockroachdb doesn’t have this problem.
I can’t, so I run Postgres. But cockroachdb doesn’t have this problem.
Backup and restore used to be a real pain. You had to write custom spark jobs previously.
Currently, it is easier with the new dataflow integration, yet it still is not very smooth. Cloud functions don't have scheduled triggers, so we ended up setting up cron jobs on our K8s to trigger backups.
Currently, it is easier with the new dataflow integration, yet it still is not very smooth. Cloud functions don't have scheduled triggers, so we ended up setting up cron jobs on our K8s to trigger backups.
I'm never sure what is better - a service that is 99% reliable and you're used to failing over to a backup system because it happens a few times a year. Or a system that is 99.99% reliable and you're ed when it goes down.
Is there a name for that dilemma and a preferred option?
I like the idea of super reilable services, but I'm also comforted by old fashioned databases I know, and have dealt with lots of live problems.
Is there a name for that dilemma and a preferred option?
I like the idea of super reilable services, but I'm also comforted by old fashioned databases I know, and have dealt with lots of live problems.
I asked previously when a CockroachDB post was on here, but didn't really get an answer to it.
Is anyone running this in prod at a significant scale and able to comment on reliability and performance?
I absolutely love CockroachDB in my toy projects, but am not yet feeling comfortable to commit to using it in prod.
Is anyone running this in prod at a significant scale and able to comment on reliability and performance?
I absolutely love CockroachDB in my toy projects, but am not yet feeling comfortable to commit to using it in prod.
From what i can see from customers page: Heroic labs and baidu
Congrats to the Cockroach crew. CockroachDB is a very interesting project, and this will make its functionality available to a wider audience.
"available at launch on both AWS and GCP"
Curious if there was some technical barrier on Azure, or if it's perhaps coming later.
Curious if there was some technical barrier on Azure, or if it's perhaps coming later.
AWS is gold standard so of course they would support it.
Cockroach labs is backed by Google Ventures and run by former Googlers so that explains GCP.
Azure may not have the most impressive tech stack, but plenty of non tech reasons for these decisions as well.
Cockroach labs is backed by Google Ventures and run by former Googlers so that explains GCP.
Azure may not have the most impressive tech stack, but plenty of non tech reasons for these decisions as well.
I think AWS and GCP both provide a better custom time synchronisation system. This is not a deal breaker because CRDB could be configured with larger time drift tolerance.
So maybe I am missing something!
So maybe I am missing something!
Does anyone have any comparison to other new sql offerings foundationdb, nuodb ect
What a dreadful name.
Understandable, but there have been so many boring threads about this that we need to just not to go there.
bb1234(3)
This really demonstrates they really care about the stability of their product, one more reason to keep recommending CockroachDB.
[0] https://github.com/cockroachdb/cockroach/issues/32048