Why Did So Many Startups Choose MongoDB?(nemil.com)
nemil.com
Why Did So Many Startups Choose MongoDB?
https://www.nemil.com/mongo/1.html
163 comments
Have you ever worked on a significant project using MongoDB?
I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.
I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.
I am working on a significant project using MongoDB. Reliability absolutely is an issue even with significant engineering resources thrown at it.
Any idea about Memsql? We are evaluating it at our company and all the info is kind of fuzzy.
Feel free to reach out to me on our public Slack channel to clear up anything that is fuzzy! I'm an engineer at MemSQL and want to know (1) how I can help and (2) how we can provide non-fuzzy information to the rest of the community.
Channel: chat.memsql.com My UN: eklhad
Channel: chat.memsql.com My UN: eklhad
MemSQL as in the distributed relational data warehouse product? If so, it's highly recommended, probably the best data warehouse solution you can run if you're not going with something managed (like redshift or bigquery).
For those actually needing scalable, resilient nosql, CouchBase ia worth evaluating.
Why wouldn't you always use something that is tried-and-true? MongoDB is relatively new compared to something like postgresql or even mysql.
Not only that but you can use PostgreSQL as a NoSQL and there are benchmarks showing that it outperforms MongoDB.
And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column.
There are a very few applications that need distributed scalability beyond what Oracle or PostgreSQL can provide, and those ones need NoSQL. They also need highly skilled competent developers who are capable of reimplementing some needed functionality of RDBMSes at scale.
And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column.
There are a very few applications that need distributed scalability beyond what Oracle or PostgreSQL can provide, and those ones need NoSQL. They also need highly skilled competent developers who are capable of reimplementing some needed functionality of RDBMSes at scale.
These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point.
There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
> These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point.
It's much much harder to design a reliable engine than a new datatype.
> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yes they are, but chances are you don't have that need. You're not Google or Facebook.
You also picked up a really bad database to talk about scalability[1]
[1] https://www.datastax.com/wp-content/themes/datastax-2014-08/...
It's much much harder to design a reliable engine than a new datatype.
> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yes they are, but chances are you don't have that need. You're not Google or Facebook.
You also picked up a really bad database to talk about scalability[1]
[1] https://www.datastax.com/wp-content/themes/datastax-2014-08/...
Postgres had hstore for a few years before that which solved similar use cases, but was only key value (no nested objects)
Plus, hstore can only handle strings, so it's actually pretty bad when weighed up against a json-store
Yep, you can't do stuff like update JSON in place until PG 9.5 (without resorting to a crazy regex, that is).
You also do have to think at least a little about your JSON structure given you can't really use an ORM to query JSONB, and the syntax for JSONB is a bit awkward at times.
You also do have to think at least a little about your JSON structure given you can't really use an ORM to query JSONB, and the syntax for JSONB is a bit awkward at times.
My guess...
SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it.
People also undervalue their data and it's integrity.
I personally do not like mongo and see zero reason for it except perhaps in some very niche use cases and even then, I'd still choose another document store over it.
SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it.
People also undervalue their data and it's integrity.
I personally do not like mongo and see zero reason for it except perhaps in some very niche use cases and even then, I'd still choose another document store over it.
Maybe I'm too old now... but SQL is relational algebra translated into some sort of code/language. I did a lot of SQL & relational algebra for my undergrad databases class [which was about 20 years ago now]. Has SQL/DBs been phased out of CS education? Just genuinely curious...
Doesn't really matter if it has or not given how many developers don't have a formal technical education.
Even 20 years ago the database class was optional for us. I didn't take it because networking was more interesting.
Nope, my college had at least 2 classes to cover databases. First one was to learn basics (although it had fun project to implement a DB engine and handle simple SQL), and second one was more about using it in real applications.
The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.
The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.
I have an intern who's bout to graduate from a state university and has taken 0 classes on the implementation of compilers or databases. Java, Java. Java.
Sounds like me 10 - 15 years ago.
I'd say don't worry: I almost never use anything I learned in school.
The Java was to shallow as was lots of other stuff.
School is just there to convince your first few employers so you can het a chance to see how it is actually done.
Oh, and to give you the credentials to stop annoying sales guys that are about to convince your boss that "you need Active Directory because your network is slow". (Guys company had botched a certain server install and tried to solve it by selling even mire stuff. )
I'd say don't worry: I almost never use anything I learned in school.
The Java was to shallow as was lots of other stuff.
School is just there to convince your first few employers so you can het a chance to see how it is actually done.
Oh, and to give you the credentials to stop annoying sales guys that are about to convince your boss that "you need Active Directory because your network is slow". (Guys company had botched a certain server install and tried to solve it by selling even mire stuff. )
I graduated from an above average university in the UK in the last 5 years and we had multiple modules covering concepts of relational databases, data warehousing, hands-on MySQL and Oracle including full-stack application projects. There was a lot of Java and little more than a Hello World in C
I finished my BS in CS 7 years ago. I took an elective class on relational databases. We spent about the first 25% of the class on relational algebra & SQL, and then used SQL fairly consistently for the remainder of the class.
Late 1990s: 68k assembler, relational algebra taught via SQL, Prolog, C, and some Ada. And astard hard discrete maths, logic, and formal methods.
Red brick Russell Group British university, not prestigious.
Red brick Russell Group British university, not prestigious.
I graduated 7-8 years ago and we did a lot of SQL so I'd surprised if that's not the case today. I think one or two SQL/relational database classes are almost a norm.
I think a web development class I took in college a few years ago spent maybe two or three class periods on SQL. Felt like enough to get by with, and learn more as you go.
[deleted]
A huge amount of people working as developers have no formal CS education whatsoever. In my company I even hear it touted as a perverse point of pride sometimes. (Kind of like, Oh you have a degree? I studied biology and I can still do the same job.)
I have a CS education, but have no real experience. So now, as a biomedical researcher, trying to rapidly prototype for medical devices and developing software is extremely daunting. I don't think these prideful people understand just how far a formal background can go to speed up the process when things get more complex.
The hope is that new technologies would provide some advantage, like prototyping speed. That was certainly true of Ruby on Rails, for example, back when it was the new kid on the block and the "tried and true" frameworks were stuff like Struts or Spring. Yes, that's not a database technology example.
If you think that every tried-and-true database is slower to develop against.
If you actually need certain features like a document store, then Postgres is less mature than MongoDB in that regard.
Have you actually studied the JSONB capabilities of PostgreSQL and related features such as partial indexes? These things can be used at scale and there is at least one company that has millions of partial indexes in production.
And you might want to look at how SalesForce has implemented a highly scalable architecture on top of Oracle RDBMS servers. That includes a document store as well as a dynamic denormalized schema layer that works real well for reporting.
Heck, I've implemented a persistent memcache server on top of SQLITE and it scaled pretty well because modern servers and network infrastructure just happen to do most of the work needed.
It is not easy to learn PostgreSQL well enough to evaluate it, but more and more people are doing so and finding that it is the ideal open source persistence solution for the vast majority of business uses.
And you might want to look at how SalesForce has implemented a highly scalable architecture on top of Oracle RDBMS servers. That includes a document store as well as a dynamic denormalized schema layer that works real well for reporting.
Heck, I've implemented a persistent memcache server on top of SQLITE and it scaled pretty well because modern servers and network infrastructure just happen to do most of the work needed.
It is not easy to learn PostgreSQL well enough to evaluate it, but more and more people are doing so and finding that it is the ideal open source persistence solution for the vast majority of business uses.
Honestly you can say any of these things about MongoDB. Their marketing team sure does. Did you know MongoDB is web scale?
I'm not insulting Postgres (actually a big fan), but Postgres added JSON support after. By that metric, document store in Postgres is less tried-and-true than MongoDB. Does that mean it's the wrong choice? No. Age of product is not always a good qualifier.
I'm not insulting Postgres (actually a big fan), but Postgres added JSON support after. By that metric, document store in Postgres is less tried-and-true than MongoDB. Does that mean it's the wrong choice? No. Age of product is not always a good qualifier.
I'll take "less mature" over "data loss" every day and twice on Sunday.
Whatever else you might say about Postgres, that community cares about the integrity of its users data, over pretty much every other consideration. I don't think the same can be said of Mongo, or any other "NoSQL" anything.
Whatever else you might say about Postgres, that community cares about the integrity of its users data, over pretty much every other consideration. I don't think the same can be said of Mongo, or any other "NoSQL" anything.
> I don't think the same can be said of Mongo, or any other "NoSQL" anything.
I feel pretty good about DynamoDB, but the development model WRT to indexes and throughput is what keeps me away.
I feel pretty good about DynamoDB, but the development model WRT to indexes and throughput is what keeps me away.
That's exactly my point. Postgres JSON support is "relatively new" (quoting OP) compared to Postgres. So clearly MongoDB is better right?
Except it's not and age of the product is a bad qualifier.
The MongoDB hate hype train is even worse than the MongoDB hype train ever was.
Except it's not and age of the product is a bad qualifier.
The MongoDB hate hype train is even worse than the MongoDB hype train ever was.
You can reimplement 95% of the "document model" with a relational database just fine, and you'll actually come out ahead given all its advantages.
Unstructured documents are very rare. Querying arbitrarily against them even more so. For the fields that do need querying in a document, put them apart and add a few indexes and you're good. That's been my experience and I've been around a while.
Unstructured documents are very rare. Querying arbitrarily against them even more so. For the fields that do need querying in a document, put them apart and add a few indexes and you're good. That's been my experience and I've been around a while.
The problem most developers who think they do really don't need a document store. They're just storing data and fields and relationships.
This post makes the assumption that MongoDB isn't doing well based on Hacker News. MongoDB the company, and product, are doing extremely well (outside of anecdotal Hacker News posts), based on objective metrics like downloads, skills, jobs, and company revenue (and the company is nearing 900 employees on linkedin).
To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
MongoDB is like the PHP of databases. I started programming PHP and currently I bash it every time I can, because it is a terrible language. But it is easy to start and do cool stuff, just like mongodb. Initial traction is more important than initial architecture, I guess.
What? Cuz it "sucks"? We all hate PHP, but it works as advertised. I don't agree with the comparison.
These days MongoDB more or less (finally) works as advertised as well. It actually has worthwhile backend options (plural!) and the new (v1 vs v0) replication protocol is actually correct. Eight or so years of development has led to the most egregious of the bugs and corner cases being rounded off.
So, it started as a shitshow on the back-end and people who had to maintain systems based on it hated it, but it got popular because it was easy to get simple systems up and running with. Then, over time, it slowly became less awful as it was substantially rewritten and large portions of it redesigned. I'd say PHP is a very, very good comparison.
So, it started as a shitshow on the back-end and people who had to maintain systems based on it hated it, but it got popular because it was easy to get simple systems up and running with. Then, over time, it slowly became less awful as it was substantially rewritten and large portions of it redesigned. I'd say PHP is a very, very good comparison.
Right; Facebook, Wikipedia, OpenStreetMap - all serious tech stacks that started and (in the case of the last two) continue to use PHP.
Reminds me of: https://slack.engineering/taking-php-seriously-cf7a60065329
Reminds me of: https://slack.engineering/taking-php-seriously-cf7a60065329
Wikipedia launched in 2001. Facebook launched in 2004.
The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy.
PHP, on the other had, was easy to get from any shared hoster.
The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy.
PHP, on the other had, was easy to get from any shared hoster.
There were many good web libraries/frameworks for Python before Django. One even had a transactional NoSQL object database.
Facebook does not "normal" PHP that much, they use host of languages, including Hack, erlang and lots of Python.
Most of their code is Hack, and Hack is a superset of PHP (last time I checked all it really adds is static typing and async)
https://news.ycombinator.com/item?id=14659729
https://news.ycombinator.com/item?id=14659729
> all it really adds is static typing and async
You make that sound like its not a big deal vs vanilla php.
You make that sound like its not a big deal vs vanilla php.
It's really not. All the sadness and broken-ness of PHP is still there.
A lot broken-ness is still left for the sake of compatibility (especially since HHVM's open-source), but Collections and type hinting make a huge difference.
Right. That's why I said "Started With".
That Slack post is terrible. I can't believe they would site concurrency as a strength of PHP...."Asynchronously curl’ing to localhost (or even another web server) provides a shared-nothing, copy-in/copy-out way of exploiting parallelism."
I almost have to wonder if this is satire
Well, Facebook and Wikipedia are using HHVM to execute their PHP code, so at least those two are not really satisfied with PHP. I bet they'd like to rewrite their code in a faster, more memory-efficient, language but are hindered by the complexity of their code base.
The OpenStreetMap website and editing API is written in Rails, not PHP. I think it was initally some other form of Ruby web application.
OSM started in 2004, but the editing API was ported to Rails in 2007ish.
OSM started in 2004, but the editing API was ported to Rails in 2007ish.
[deleted]
Why is PHP a terrible language? It powers a vast part of the internet including sites huge, tiny, and everything in between.
It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
This lack of insight is why PHP developers have lower average salaries than non-PHP developers.
If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP.
Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....
If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP.
Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....
You're not wrong, but those articles are 5+ years old and PHP has changed A LOT in recent years — especially since the release of v7.
Still, that's not going to do much to change public perception, and I guess that's somewhat fair. PHP has earned it's reputation, even if many (but not all) criticism are now largely dated.
I'm just happy I switched to working on other languages, aside from the occasional Laravel-based side projects.
Still, that's not going to do much to change public perception, and I guess that's somewhat fair. PHP has earned it's reputation, even if many (but not all) criticism are now largely dated.
I'm just happy I switched to working on other languages, aside from the occasional Laravel-based side projects.
> PHP has changed A LOT in recent years — especially since the release of v7.
I was curious and wanted to see if that is the case. So I checked for the most basic thing. The arrow notation for lambdas. Surely they must've added that by now. Surely they must've added that before Java did. Nope, still RFC. Now I wonder if it's idiomatic to pass those "long form" lambdas around when writing in functional style or is it something nobody does. In any event this little discovery makes me think parent's sentiment seems to be still justified.
I was curious and wanted to see if that is the case. So I checked for the most basic thing. The arrow notation for lambdas. Surely they must've added that by now. Surely they must've added that before Java did. Nope, still RFC. Now I wonder if it's idiomatic to pass those "long form" lambdas around when writing in functional style or is it something nobody does. In any event this little discovery makes me think parent's sentiment seems to be still justified.
I guess that's one way of judging a software language's progress, sure.
Thanks, that is absolutely worth pointing out. But there is only so much you can do to fix languages when they are designed with problems initially and you want to keep old programs working. I have the same concerns about ES6 BTW, although JavaScript foundations are arguably less flawed overall (excepting a few areas that are just bat shit crazy).
Is public perception the reason you are happy to be mostly using something else now, though?
Is public perception the reason you are happy to be mostly using something else now, though?
It's certainly nice when talking in developer circles, but no, public perception is not the primary reason — it's more of a bonus :)
I really enjoy some of the more flexible syntax and design choices present in some other languages. It also help that various job requirements over the year have required that I build a broader skillset. Finally, the growth I've experienced by stepping outside my comfort zone the past few years is probably the best benefit, which is a good experience regardless of which language(s) you start with.
In the end, however, as we both know, most decisions in software development are an exercise in managing tradeoffs. I guess the core of my argument is that, in the past few years, PHP has done a great job at improving its balance of tradeoffs.
I really enjoy some of the more flexible syntax and design choices present in some other languages. It also help that various job requirements over the year have required that I build a broader skillset. Finally, the growth I've experienced by stepping outside my comfort zone the past few years is probably the best benefit, which is a good experience regardless of which language(s) you start with.
In the end, however, as we both know, most decisions in software development are an exercise in managing tradeoffs. I guess the core of my argument is that, in the past few years, PHP has done a great job at improving its balance of tradeoffs.
Programmers discussing languages is like carpenters discussing screwdrivers.
If you don't know, then I can't tell you :P
Seriously, the biggest problem with php is that fundamentally it's a templating language. One thing I've seen with templating libraries in general, is that they are meant to take a template and a data structure and display output based on these 2, but what ends up happening is they end up converging on turing completeness. It starts with the need for some kind of looping structure in the template. And then someone needs some random logic that is easier to do in the template than before the model, and so another feature gets added to the templating engine. And so it goes, things get kind of bolted on, without a grand vision, and you end up with an API that is inconsistent, where the order of inputs is never the same (is the first parameter haystack or needle??).
Also, the basic shape of the code that tends to get written in PHP is heavily influenced by it's being a templating language. PHP applications tend to end up having a lot of code in the template, that should be done before rendering the model. Rather than a sane 2 pass system, where all the data is gathered and the logic handled to produce a simple data structure that is passed to a very simple template, you end up with crazy monstrosities where there's opening and closing <?php> tags everywhere. And you render part of the page, then grab some more data, then render some more page, then maybe do a curl to a 3rd party vendor, then render some more of the page, and it's actually very hard to keep switching mental state to match the context of the code you're looking at.
Anyhoooo, you're getting downvoted because there's lots of literature on why PHP is a terrible language. I'm only replying because it's sort of cathartic, since I maintain a wordpress codebase, and any time I have to interact with core, instead of being able to sanely intercept the underlying data, modify it then pass it along in one place, I have to constantly add lots of random hooks in lots of random places to catch the data just before it's being rendered. It's a total mess, and far more complicated than it needs to be (though there is some debate whether this style made it more approachable from a novice plugin development perspective).
Seriously, the biggest problem with php is that fundamentally it's a templating language. One thing I've seen with templating libraries in general, is that they are meant to take a template and a data structure and display output based on these 2, but what ends up happening is they end up converging on turing completeness. It starts with the need for some kind of looping structure in the template. And then someone needs some random logic that is easier to do in the template than before the model, and so another feature gets added to the templating engine. And so it goes, things get kind of bolted on, without a grand vision, and you end up with an API that is inconsistent, where the order of inputs is never the same (is the first parameter haystack or needle??).
Also, the basic shape of the code that tends to get written in PHP is heavily influenced by it's being a templating language. PHP applications tend to end up having a lot of code in the template, that should be done before rendering the model. Rather than a sane 2 pass system, where all the data is gathered and the logic handled to produce a simple data structure that is passed to a very simple template, you end up with crazy monstrosities where there's opening and closing <?php> tags everywhere. And you render part of the page, then grab some more data, then render some more page, then maybe do a curl to a 3rd party vendor, then render some more of the page, and it's actually very hard to keep switching mental state to match the context of the code you're looking at.
Anyhoooo, you're getting downvoted because there's lots of literature on why PHP is a terrible language. I'm only replying because it's sort of cathartic, since I maintain a wordpress codebase, and any time I have to interact with core, instead of being able to sanely intercept the underlying data, modify it then pass it along in one place, I have to constantly add lots of random hooks in lots of random places to catch the data just before it's being rendered. It's a total mess, and far more complicated than it needs to be (though there is some debate whether this style made it more approachable from a novice plugin development perspective).
You described exactly what I'm experiencing. I've had this awkward feeling that was hard to describe in my current work and you were able to express my exact thoughts in words. Thanks for sharing
Way back when I started reading HN, one thing that really struck me was seeing PHP discussed in a practical fashion rather than merely bashing it whenever it's mentioned as was the case in Proggit those days (often, the bashers obviously having very little experience building software).
By that time I was doing most of my coding in Ruby and Python, but this businesslike approach to PHP made me realize this is a place where people really get their hands dirty and build things, rather than care about fads.
Things have really changed, the tone & level of discussion here are now very different. Seems like PHP is kind of a canary in the coal mine in that regard.
By that time I was doing most of my coding in Ruby and Python, but this businesslike approach to PHP made me realize this is a place where people really get their hands dirty and build things, rather than care about fads.
Things have really changed, the tone & level of discussion here are now very different. Seems like PHP is kind of a canary in the coal mine in that regard.
What are you talking about? PHP has been rightfully bashed ever since it was created. If anything the bashing has substantially gone down in the past years as sane array notation has been added and a lot of weirdness has gone away.
It's not "in vogue". There's no trend here, the PHP hate has been very consistent over time.
Personally, having had to write a couple of Mediawiki plugins and an extension for a rubbish CMS called TYPO3, my experience so far is that the PHP hate is very much justified.
Personally, having had to write a couple of Mediawiki plugins and an extension for a rubbish CMS called TYPO3, my experience so far is that the PHP hate is very much justified.
> It powers a vast part of the internet including sites huge, tiny, and everything in between.
It's more like those vast parts of the internet are working despite PHP. Why do you think Windows is powering vast numbers of desktops? It's not for its technical excellency. ;)
It's more like those vast parts of the internet are working despite PHP. Why do you think Windows is powering vast numbers of desktops? It's not for its technical excellency. ;)
Can't speak to current popularity, but I think the overall critique is spot on. It became really popular riding a combination of up-front friendliness and the No-SQL hype bubble. The next morning, a lot of folks started feeling the hangover, whether from actual problems with Mongo or a realization that they did, in fact, need a relational DB.
To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype.
It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype.
It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
I don't know. I was just in a meeting where a team announced that they would need a ton of money and time to move off of MongoDB. The VP reminded them of when they had chosen to move to MongoDB years ago and asked them what happened to all of their promises about how great MongoDB would be. They responded that the promise of MongoDB and the reality are very different and MongoDB has turned out to be a disaster.
> objective metrics like downloads, skills, jobs, and company revenue
Downloads can be repeats, no? How to discern real number of productive installs?
Skills aren't objective, are they?
Revenue, great...but they won't disclose it.
Downloads can be repeats, no? How to discern real number of productive installs?
Skills aren't objective, are they?
Revenue, great...but they won't disclose it.
I understand the importance of well thought out back-end architecture, and that it isn't my realm. I just have to say that I really enjoy building things in the MEAN stack, and had fun learning MongoDB.
For a long time all I had ever done was relational, but then my company started using Azure table storage for a lot of projects and I became familiar with other models. Then I discovered Mongo while catching up on Angular, and enjoyed using it with the mostly small apps and prototypes I build. I don't know it well enough to try and support it's use on a large scale. After seeing how poorly designed the data is for so many companies out there, it's not a surprise to hear poor choices were made (and probably some poor execution).
For a long time all I had ever done was relational, but then my company started using Azure table storage for a lot of projects and I became familiar with other models. Then I discovered Mongo while catching up on Angular, and enjoyed using it with the mostly small apps and prototypes I build. I don't know it well enough to try and support it's use on a large scale. After seeing how poorly designed the data is for so many companies out there, it's not a surprise to hear poor choices were made (and probably some poor execution).
But MongoDB's own cite cites Indeed.com Job Trends data only through 2012, on the basis that Mongo's growth is a reason to use it.
Current Indeed.com Job Trends data actually show better numbers than the 2102 stats, but nonetheless show MongoDB jobs in decline.
https://www.indeed.com/jobtrends/q-MongoDB.html
Current Indeed.com Job Trends data actually show better numbers than the 2102 stats, but nonetheless show MongoDB jobs in decline.
https://www.indeed.com/jobtrends/q-MongoDB.html
[deleted]
Back in 2012 I worked at a startup and we had a few backend data stores (Vertica, Postgres, Redis, Big Table, MySQL, Cassandra, ElasticSearch, Hadoop).
When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization).
At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems.
That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization).
At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems.
That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
I think the worst part about MongoDB in that timeframe was the poor CPU utilization since it was single threaded.
In order to leverage the multiple CPUs in a box you'd need to run multiple instances of mongo on the same machine and we designed a whole system around sandboxing instances.
But then you'd run into memory allocation issues and how badly it would handle indexing and handle running out of memory, which mostly involved crashing.
We ended up having to run about 100 bare-metal servers to support what would probably have been a level of queries about 10 PostgreSQL boxes could handle easily.
Truly a terrible DB.
In order to leverage the multiple CPUs in a box you'd need to run multiple instances of mongo on the same machine and we designed a whole system around sandboxing instances.
But then you'd run into memory allocation issues and how badly it would handle indexing and handle running out of memory, which mostly involved crashing.
We ended up having to run about 100 bare-metal servers to support what would probably have been a level of queries about 10 PostgreSQL boxes could handle easily.
Truly a terrible DB.
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft.
http://bit.ly/2uP3Obi
http://bit.ly/2uP3Obi
But... Kafka isn't a database?...
No, it's not. We were trying to wrap our own system around Mongo to do the job of Kafka which is why I said that.
I have been paid by two companies to remove Mongo. One that was hosted at the Mongo HQ. (The mongo HQ guys are really nice). At small startups, things are on fire and people need answers quick. "How many users tried this, how many times was feature x used," on and on. If it is in mysql, well i just setup a vpn and a read mysql access, done -- they can query away.
If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
I wrote this series over several months and am happy to answer questions.
Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr
As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.
Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr
As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.
I think Enterprise folks do not appreciate the flexibility afforded by NoSQL . For my startup I started off on PostGres SQL with Hibernate and it worked great ... until we first met our paying customers. They needed data elements that I had not envisioned and so we ended up doing Schema updates.. again and again . Since we used ORM (hibernate), that also meant new hibernate code generation . After a while this became very tedious.
I have a background in enterprise s/w development; we always lock down on most of our requirements before we start writing code. RDBMS is fantastic in this scenarios. In my startup, we build something light and show it to bunch of prospects and iterate. MongoDB fits this mode of s/w development for us. Sure there are gotchas and other issues but they have not affected us yet.
I have a background in enterprise s/w development; we always lock down on most of our requirements before we start writing code. RDBMS is fantastic in this scenarios. In my startup, we build something light and show it to bunch of prospects and iterate. MongoDB fits this mode of s/w development for us. Sure there are gotchas and other issues but they have not affected us yet.
The gotcha is going to come when your schema has dozens of fields whose existence is uncertain and they begin generating runtime errors when your code expects them to exist.
You can do NoSQL with Postgres...
You guys are discussing MongoDB from 2009. The database has evolved since then.
- There is a new replication protocol for replica sets, improving the election process.
- WiredTiger has been introduced and became the default storage engine.
- Journaling guarantees durability.
- Sharding architecture has changed. Config Servers are now deployed as Replica Sets and some of the tasks that used to be managed by mongos now are managed by the config servers.
- Storage efficiency has been added through compression.
- Security has been improved with encryption at rest.
- The aggregation framework has more options for BI.
- There are a variety of different indexes that can be created to improve query performance.
- and finally, stop saying MongoDB is schemaless because it is not. There is a schema in place and this schema is flexible enough to be polyform.
These are features available today in 3.4. You might have issues in the past, but if you really understand the newest architecture, you are capable of seeing how much MongoDB has grown since 2.4.
- There is a new replication protocol for replica sets, improving the election process.
- WiredTiger has been introduced and became the default storage engine.
- Journaling guarantees durability.
- Sharding architecture has changed. Config Servers are now deployed as Replica Sets and some of the tasks that used to be managed by mongos now are managed by the config servers.
- Storage efficiency has been added through compression.
- Security has been improved with encryption at rest.
- The aggregation framework has more options for BI.
- There are a variety of different indexes that can be created to improve query performance.
- and finally, stop saying MongoDB is schemaless because it is not. There is a schema in place and this schema is flexible enough to be polyform.
These are features available today in 3.4. You might have issues in the past, but if you really understand the newest architecture, you are capable of seeing how much MongoDB has grown since 2.4.
But at this point that's all pretty much moot because competitors have caught up and there's hardly an advantage over Postgres' JSONB fields, which builds up on top of all the features Mongo will likely never have (transactions I'm looking at you).
I use Mongo & its been working great. Our data is not relational (imagine you were tasked with building "Powerpoint" on the web).
Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over.
With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works".
Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over.
With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works".
Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
Of course I don't know your product, but when I imagine Powerpoint-as-a service, do users not relate to decks which relate to slides which relate to the content on those slides? Content itself of course would probably be a pretty flexible tree structure.
An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me.
And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast.
If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product.
HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen.
As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me.
And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast.
If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product.
HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen.
As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
> Get me all the content of slide Y of deck XXXX
But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.
But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.
> would make me itching for indices and foreign constraints
Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.
> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"
Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.
But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.
But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.
> would make me itching for indices and foreign constraints
Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.
> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"
Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.
Interesting article, and I look forward to the other 2 parts. This was well-researched and balanced.
MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed.
Someone else commented about the HN bias, which we can't deny exists.
I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today.
Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ...
Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained.
I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed.
Someone else commented about the HN bias, which we can't deny exists.
I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today.
Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ...
Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained.
I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
I think the short answer is the same today as it was back in those days. A person once put it to me simply: "programmers are afraid of databases."
It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper.
Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper.
Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
> programmers are afraid of databases.
Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization.
> Add the idea that there is no need for a DBA
With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well.
> encourage developers to see Mongo as a magic data storage and nothing more
In modern frameworks, this tends to be true no matter what the data store is.
Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization.
> Add the idea that there is no need for a DBA
With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well.
> encourage developers to see Mongo as a magic data storage and nothing more
In modern frameworks, this tends to be true no matter what the data store is.
Mongo succeeded for three reasons:
1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks.
2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails.
3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials.
These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks.
2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails.
3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials.
These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
Not so different in some ways from the rise of MySQL itself.
In point #2 substitute PHP for Node and RoR.
In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host.
The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
In point #2 substitute PHP for Node and RoR.
In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host.
The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
MongoDB is an interesting database for sure. It seems that they made a deliberate product decision to optimize developer experience and ease of use over things like consistency and database joins. Frankly, I think it's working pretty well for their business.
I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services.
People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services.
People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
We chose it because the lead developer liked it, and when I did my research I didn't find any glaring issues because it was too new.
The lead dev had never actually used it, but liked the idea of not having to come up with a schema and he really liked JSON as he was very familiar with it.
By the time we were done, we were easily one of the top 5 users of MongoDB at scale and it was responsible for 9 out of 10 of our outages due to various bugs or optimization issues.
The lead dev had never actually used it, but liked the idea of not having to come up with a schema and he really liked JSON as he was very familiar with it.
By the time we were done, we were easily one of the top 5 users of MongoDB at scale and it was responsible for 9 out of 10 of our outages due to various bugs or optimization issues.
I'm not a back-end dev strictly. I played with mongo on some side projects back when it was all hyped up. One thing that appealed to me was the on-boarding process for a non "database guy". I got lots of small side projects up and running very quickly and the concepts made sense. Similar to jQuery being easy to understand and popular to newcomers but you later learn it isn't always the best tool for the job.
Any criticism of MongoDB that does not acknowledge WiredTiger and the plethora of positive changes that made MongoDB much, much better in the 3.0, 3.2 and 3.4 releases is criticism that is not fully thought out.
It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
In the end it doesn't really matter. I feel the article is an educational one, but as it's the chance for us to take our sides of the fence in praising and criticising, we'll still do it anyway.
I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
> I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
You realize that this sounds like more of the same hype to anyone who was burned by the overpromising in the 1.x and 2.x days? Whether your statement about 3.x is true or not, the MongoDB people dug their own hole that they need to try extra hard to get out of, and so far it's not enough.
[deleted]
It's not just because of hype that teams choose Mongo.
The flexibility of a schemaless db is pretty helpful in a startup scenario, where the requirements drastically change all the time.
Yes, migrations work for RDBMS too, but with Mongo you only write migrations for data changes, not schema changes - which means a lot less migrations.
The option of having documents with embedded objects/lists makes it a pretty natural mapping to Domain Driven Design concepts, like aggregate roots + aggregates. One might even argue that while following DDD, it's preferable to handle most rules at the application layer and use the database simply as a performant, low-maintenance storage service. I know some folks here might view this approach as a sin, but it works pretty well too, especially for smaller teams without a dedicated DBA.
Maybe it's just me that haven't ran large enough systems to find the kind of issues with mongo that people always rant about, idk. Honest question: considering all these advantages above (from a developer's point of view), are there good reasons for choosing something like PostgreSQL over Mongo for typical business apps?
Yes, migrations work for RDBMS too, but with Mongo you only write migrations for data changes, not schema changes - which means a lot less migrations.
The option of having documents with embedded objects/lists makes it a pretty natural mapping to Domain Driven Design concepts, like aggregate roots + aggregates. One might even argue that while following DDD, it's preferable to handle most rules at the application layer and use the database simply as a performant, low-maintenance storage service. I know some folks here might view this approach as a sin, but it works pretty well too, especially for smaller teams without a dedicated DBA.
Maybe it's just me that haven't ran large enough systems to find the kind of issues with mongo that people always rant about, idk. Honest question: considering all these advantages above (from a developer's point of view), are there good reasons for choosing something like PostgreSQL over Mongo for typical business apps?
I'm not entirely convinced that MongoDB is better suited or a natural mapping to DDD concepts in practice, it's just more convenient for developers who want to build things fast.
My thoughts on this pattern are to still use a robust, reliable db (Postgres) behind a message queue for the event store. Postgres does almost all of the things I need with the reliability that I need while letting me sleep at night.
An event store absolutely must be the most reliable part of the infrastructure, IMO.
My thoughts on this pattern are to still use a robust, reliable db (Postgres) behind a message queue for the event store. Postgres does almost all of the things I need with the reliability that I need while letting me sleep at night.
An event store absolutely must be the most reliable part of the infrastructure, IMO.
"just more convenient for developers who want to build things fast" - that's a compelling advantage for Mongo, isn't it?
I agree a robust, reliable db is super important - but isn't Mongo too, at this point? Even if it wasn't back in 2012? I thought most of the reliability issues and bad defaults were fixed by now...
I agree a robust, reliable db is super important - but isn't Mongo too, at this point? Even if it wasn't back in 2012? I thought most of the reliability issues and bad defaults were fixed by now...
My (in)experience with Mongo leaves me with a certain existential dread that I just can't shake. Also for that sort of thing these days I'd be more likely to use ElasticSearch than anything (this absolutely terrifies me actually, but I'm seeing companies do this) and eat the extra infrastructure expense. Serves the same function and gives me insanely powerful tools to process the data without having to write any code.
In my planning for problem domain changing, personally I'm going to be writing a (relational) schema for the new data model anyway, which will be Postgres, and playing my event store over it and doing one-way writes to it going forward anyway. I'd rather reduce the mental maintenance overhead of two different databases to one.
In my planning for problem domain changing, personally I'm going to be writing a (relational) schema for the new data model anyway, which will be Postgres, and playing my event store over it and doing one-way writes to it going forward anyway. I'd rather reduce the mental maintenance overhead of two different databases to one.
Interesting. I've always thought event store was an interesting pattern, but not something as a default architecture for typical business apps, probably because of the added complexity in maintenance.
If you're building your relational db based on the event store, I'd imagine even something simpler like Mongo would do it, but I agree that having a single db to maintain is an important advantage.
If you're building your relational db based on the event store, I'd imagine even something simpler like Mongo would do it, but I agree that having a single db to maintain is an important advantage.
Schema migrations during rapid growth are usually of the ADD COLUMN type which is cheap & easy. Data migrations are about the same amount of effort either way. If anything, requiring explicit schema migrations forces you to think more carefully about your schema design decisions and saves headache down the road, as the article mentions Etsy (and many others) encountered.
I still think MongoDB is a good database. Conceptually, It's not as good as RethinkDB but it's not too far off. It's good to not have to deal with SQL and horrible ORMs anymore.
TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
> It's good to not have to deal with SQL...
I will never understand this attitude.
I will never understand this attitude.
I don't understand what you're saying MongoDB solved. Mongo has ORMs too like Mongoose.
Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
Mongoose is a lot simpler and less brittle than other ORMs in SQL land. That said, for most use cases, you don't need to use Mongoose.
When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions.
With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions.
With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
Looking at the table from 10th Gen, I believe it is misleading, at least in relation to Edmunds. I worked there, and Oracle (which was the source of truth) was replaced by PostgreSQL.
Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost.
I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went.
I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives.
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost.
I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went.
I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives.
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
We are also one of those companies that bought into the promise (or hype) in 2011 and gave MongoDB a spin. After some very successful tests, I had long conversations with my CTO on why we should deploy MongoDB in prod and make it our primary OLTP store. Yes, MongoDB had it's challenges in 1.6 and it required some "faith". But it's 2017 now and MongoDB totally delivered on it’s vision, at least for us.
Since 2012, that's when we upgraded to 2.0, we had 10 min of total downtime. That's 99.999% uptime in 5 years. Year after year, 99.999% uptime. And that 10 min is the total for scheduled (btw, we don't have scheduled downtime for our MongoDB clusters since there is no need for that) and un-scheduled downtime. We never take the service down. We perform all maintenance operations during regular business hours with no impact to end-users at all. So much about reliability!
MongoDB as a product has bugs. What a surprise, what software product doesn't. But it's about you community (we are running the community edition) supports the product. We recently discovered an issue as well causing the primary to crash. As designed, another member in the repl set took over and there was no downtime at all. Even though we don't have a support contract, the issue was acknowledged by MongoDB in hours and fixed in a couple of days. I wish we had that type of support for products where we do have a support contract!
And last but not least, if you can't figure out how to successfully use MongoDB, for god sake, take some free training classes on MongoDB University.
MongoDB might not be a good fit for every use-case, but it's already an awesome platform for an incredible number of applications. Polyglot environments is the keyword here. SQL has it’s advantages and disadvantages and so does NoSQL. Use the best tool for the problem. One size fits all doesn't work anymore and MongoDB is clearly the best and most popular document store on the market. And even better, other platforms like postgres have acknowledged that the idea of storing JSON has its merits and have implemented its own version of that idea. However, IMO, if your use-case is a good fit for a document store, use MongoDB. We have done so and couldn't be happier with that decision.
What NoSQL databases are people using today when they need something ACID compliant? CouchDB?
NoSQL originally meany to not use SQL as the query language
However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
ACID and relational are orthogonal. You can have ACID properties without a relational model. Much of the early literature on the subject merely discusses databases in general rather than any particular type. Here's an example from Jim Grey:
http://jimgray.azurewebsites.net/papers/thetransactionconcep...
http://jimgray.azurewebsites.net/papers/thetransactionconcep...
> NoSQL originally meany to not use SQL as the query language
I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
For us, the reason to run with MongoDB in 2009 were twofold: a) for one of our products which was consuming the Twitter gardenhose, unverified writes were a blessing. We got away with a single dedicated server running mongodb (and it was not a particularly expensive server). I would need to do serious research again whether there's a better solution for this today. It was using Stanford NLP to highlight breaking news before anyone realized. We never got a single subscriber because every newsdesk that saw it wanted to acquire us immediately LOL. Eventually one did. b) For another, the ability to store and index complex documents which in SQL would've needed JOINs and you couldn't index across JOINs at least in MySQL and our previous experience in PostgreSQL vs MySQL support didn't favor PostgreSQL at the time at all. It worked -- although it didn't have a journal back then. We were ready for a failure but the failure never came. The landscape today is much different with PostgreSQL and MySQL and SQLite all embracing JSON storage and indexing on expressions are there for all three.
I find it difficult to seriously address comments to this thread. People still referencing 10gen and talking about data points from 5-10 years ago (2009, 2012, 2013).
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft.
http://bit.ly/2uP3Obi
MongoDB has many built in features that are extremely attractive for building modern applications. Built in HA & DR, horizontal scalability, multiple storage engines, rich indexing and query capabilities, enhanced BI features in aggregation framework, and many more...
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft.
http://bit.ly/2uP3Obi
MongoDB has many built in features that are extremely attractive for building modern applications. Built in HA & DR, horizontal scalability, multiple storage engines, rich indexing and query capabilities, enhanced BI features in aggregation framework, and many more...
Not to sound flippant but the reason why so many startups choose mongodb is because the problems they are working on are better suited to be addressed with a document-oriented database than relational database.
And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
What are those problems exactly?
Come on, we all know why: https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36C...
That made me lol!
This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.
No wonder the whole javascript stack is a complete and utter mess!
This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.
No wonder the whole javascript stack is a complete and utter mess!
[deleted]
Hype and lack of critical thinking abolutely had their role, but mostly because it's so much easier than a relational db.
Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts.
The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server.
Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping.
Who cares about data integrity or technical debt?
Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts.
The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server.
Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping.
Who cares about data integrity or technical debt?
It has most of that. It's not that RDBMS are hard and schemaless convolutions are easier.
Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection.
I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection.
I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
I've assessed a fair number of companies who use Mongo in some way or another and the I always ask the rationale for the decision. Surprise surprise, it's usually "we chose it because it was easy to start with".
My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
In case of Edmunds, the rationale was that their contractors previously used Mongo and were more familiar with, and upcoming 2.6 version will have some "new cool features".
Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1].
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1].
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
Obligatory read when talking MongoDB https://aphyr.com/posts/322-jepsen-mongodb-stale-reads and follow-up https://jepsen.io/analyses/mongodb-3-4-0-rc3.
Ensured me to never use this for production scenarios. No matter how 'hyped' or 'convenient' it is. Yaiks.
Ensured me to never use this for production scenarios. No matter how 'hyped' or 'convenient' it is. Yaiks.
As a person who just recently turned down an offer to write an ArangoDB book, Mongo is not worth it. I think most needs are met between Arango and PostgreSQL. Arango provides a scalable NoSQL option over PostgreSQL (let's see what version 10 holds). PostgreSQL provides a solid, efficient SQL solution.
Mongo provides no benefits of transactions or solid design. If you want either, look to Arango or PostgreSQL.
Mongo provides no benefits of transactions or solid design. If you want either, look to Arango or PostgreSQL.
There were so many blog posts about how MongoDB was the future and SQL/relational dbs were the past.
I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
Our team had little experience with backends in 2013 and a fear that it will be hard to scale game servers in case of success (players hate lags and downtimes).
So we followed the hype of Node.js and MongoDB because everybody was saying how it's easy to scale them compared to the established tech.
So we followed the hype of Node.js and MongoDB because everybody was saying how it's easy to scale them compared to the established tech.
Similar questions can be asked
> Why do so many developers use map/reduce instead of the database for select, pluck, math?
> Why do so many developers use length instead of counting sql?
The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.
> Why do so many developers use map/reduce instead of the database for select, pluck, math?
> Why do so many developers use length instead of counting sql?
The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.
Because they have great docs. At least that's why I chose them for my (now defunct) startup back in 2013. I wasn't familiar with databases and their docs were extremely approachable and easy to understand and navigate, with lots of useful examples.
Reminds me of the keynote DHH gave at RailsConf this year. Despite the stories that we like to tell ourselves about picking the right tool for the job, we're just as prone to chasing the shiny new thing as all the other humans.
Who's "we"? I have yet to see a high quality technical argument for using a number of technologies that have come along the last 5-8 years or so in most cases. Too often "scalability" and "modern stack" are used as an excuse to tinker with the new shiny far out of bounds of their proper use. I have a reputation as being the opposition any time adoption of something new comes up. It's rarely necessary.
I've seen a few comments mention RethinkDB ... what are the advantages over MongoDB?
I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.
I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.
The company got bankrupted and they set up an org to fund its development. (Sagemath moved to Postgresql from RethinkDB after the company got sold. The guy behind sage said it was better in his blog about the switch)
Thanks; here’s my blog post: http://blog.sagemath.com/2017/02/09/rethinkdb-vs-postgres.ht...
Incidentally, http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
Incidentally, http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
The RethinkDB company went out of business last year, and unfortunately development has basically stopped, as you can see from the commits to their repo: https://github.com/rethinkdb/rethinkdb/graphs/contributors
The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb
RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today.
I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb
RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today.
I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
Marketing aside, MongoDB is just really easy to pick up and use. In a small project, its weaknesses don't really show up (at least in my limited experience), if it gets big you often end up rebuilding anyway.
Anyone with average to more DB experience knows there is no one-stop shop. DBs are about trade-offs and we need to choose the right ones for our application(s)
Knowledge does not equal wisdom. Perhaps it was young folks with lots of knowledge didn't see the wisdom in using battle-hardened, boring technology.
It isn't an age thing. There are mid-life engineers where I work who are constantly trying to force the latest fad in tech into our engineering organization.
Anyone with proper unbiased DB knowledge knows without trying or reading anything that it is hype. DBs are all about trade-offs you can't have it all
Marketing & ease of use (for example, if you try to insert into a collection that does not exist, it's automatically created for you).
Because it's not MySql. "We can't use our dad's database!"
Really good marketing, mongo's pitch was "use this if you want to be Innovative"
[deleted]
to avoid schema migrations
And in turn put the schema (actually multiple versions of it) in your application making it more complex.
Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay.
JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux.
[1] https://en.wikipedia.org/wiki/XML_database
[2] https://en.wikipedia.org/wiki/Database_model#Object-oriented...
Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay.
JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux.
[1] https://en.wikipedia.org/wiki/XML_database
[2] https://en.wikipedia.org/wiki/Database_model#Object-oriented...
I'd rather keep the logic in my app than have planned downtime for schematic migrations
I'd wanted to lift the burden of writing ORM layers and dealing with relational algebra and focus on persisting my intrinsic application models. I'd studied XML and other semi-structured web data at the graduate level, but was frustrated there wasn't a technology that was meeting that need. Companies like Microsoft and Oracle were adding XML functionality to their databases, but because it was bolted on an already constrained structure, it failed to address the mangling that relational structures impose.
I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows.
There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch?
Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings.
When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful.
Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later.
But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB.
I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box.
And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition.
I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.
I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows.
There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch?
Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings.
When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful.
Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later.
But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB.
I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box.
And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition.
I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.
10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did)
Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric.
Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding.
This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.