Open Distro for Elasticsearch(opendistro.github.io)
opendistro.github.io
Open Distro for Elasticsearch
https://opendistro.github.io/for-elasticsearch/
325 comments
AWS explains their rationale at https://aws.amazon.com/blogs/opensource/keeping-open-source-... . Here's a snippet"
" Unfortunately, since June 2018, we have witnessed significant intermingling of proprietary code into the code base. While an Apache 2.0 licensed download is still available, there is an extreme lack of clarity as to what customers who care about open source are getting and what they can depend on. For example, neither release notes nor documentation make it clear what is open source and what is proprietary. Enterprise developers may inadvertently apply a fix or enhancement to the proprietary source code. This is hard to track and govern, could lead to breach of license, and could lead to immediate termination of rights (for both proprietary free and paid). Individual code commits also increasingly contain both open source and proprietary code, making it very difficult for developers who want to only work on open source to contribute and participate. In addition, the innovation focus has shifted from furthering the open source distribution to making the proprietary distribution popular. This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path. "
" Unfortunately, since June 2018, we have witnessed significant intermingling of proprietary code into the code base. While an Apache 2.0 licensed download is still available, there is an extreme lack of clarity as to what customers who care about open source are getting and what they can depend on. For example, neither release notes nor documentation make it clear what is open source and what is proprietary. Enterprise developers may inadvertently apply a fix or enhancement to the proprietary source code. This is hard to track and govern, could lead to breach of license, and could lead to immediate termination of rights (for both proprietary free and paid). Individual code commits also increasingly contain both open source and proprietary code, making it very difficult for developers who want to only work on open source to contribute and participate. In addition, the innovation focus has shifted from furthering the open source distribution to making the proprietary distribution popular. This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path. "
When you publish open source, this is what you sign up for. As a publisher of open source software, I think "open core" does a huge disservice to the community as it feels deceptive. While there is no denying that creators needs to be paid, communities thrive on freedom and openness, and dual licensing takes that away.
Companies that publish open source must realize that they cannot make money from the "software". Open source gives companies a brand, that they can then leverage to do other services like support, consulting, hosting, merchandise, events, etc.
There was a time when service companies wanted to move into products because of high margins and artificial scarcity, but enough open source will ensure that software licensing is not a sustainable model to seek rent, and product companies must also move to services to be able to grow and sustain.
Companies that publish open source must realize that they cannot make money from the "software". Open source gives companies a brand, that they can then leverage to do other services like support, consulting, hosting, merchandise, events, etc.
There was a time when service companies wanted to move into products because of high margins and artificial scarcity, but enough open source will ensure that software licensing is not a sustainable model to seek rent, and product companies must also move to services to be able to grow and sustain.
Reminds me exactly of an article I read this once. It's about the concept of companies commoditizing their product's complement.[1] AWS sells hosting/infra, for them lowering the entry barrier is beneficial. (I'm not taking sides just a thought)
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
I think Amazon does a much bigger disservice.
> but enough open source will ensure that software licensing is not a sustainable model to seek rent
So your solution to this is to just make it so no one can make money doing work.
Except Jeff. Jeff gets all the money.
> but enough open source will ensure that software licensing is not a sustainable model to seek rent
So your solution to this is to just make it so no one can make money doing work.
Except Jeff. Jeff gets all the money.
Looking at it rationally, is furthering open source admirable when it's done by a community activist, and loathsome when it's done by someone who has a business interest in the exact same outcome? Why isn't the end result the most important part of the equation? If a stronger, open sourced Elasticsearch comes out of this, why is that a bad thing because it was backed by Amazon?
Just be honest, and don't pretend to be open source, when you are attempting to harness goodwill towards open source in order to make a buck. As pointed out, ES builds on the work of hundreds of other contributors to other open source projects. Is Elastic compensating those developers?
> don't pretend to be open source
In my mind, a company that makes lots of free contributes to open source is an open-source-friendly company, but this thread and the other on the Elastic post made me realize just how many people think that a True Open Source Company (whatever that means) should prefer to go out of business to dealing in proprietary software.
I personally don't care for any of these puritanical viewpoints that deny economic realities (there's only so much a person or a collective can afford to give away for free, money doesn't grow on trees, etc). Open source is good, but thriving, sustainable open source depends on a symbiotic relationship with for-profit institutions.
In my mind, a company that makes lots of free contributes to open source is an open-source-friendly company, but this thread and the other on the Elastic post made me realize just how many people think that a True Open Source Company (whatever that means) should prefer to go out of business to dealing in proprietary software.
I personally don't care for any of these puritanical viewpoints that deny economic realities (there's only so much a person or a collective can afford to give away for free, money doesn't grow on trees, etc). Open source is good, but thriving, sustainable open source depends on a symbiotic relationship with for-profit institutions.
Well said. The issue is most of the core devs don't want to touch "support, consulting, hosting". It's a mental block. They think they can skip around doing only the "cool" stuff.
Many OSS companies have no problem with that. It's more of a problem that users demand features and consulting without rendering payment.
The commercial entities associated with the most popular open source technologies are taking measures to protect themselves from Amazon simply taking their software and turning it into a service, whether it's introducing proprietary components or new licensing. Wouldn't be surprised if we saw AWS do something similar for Redis, MongoDB, Kafka, etc.
> Wouldn't be surprised if we saw AWS do something similar for Redis, MongoDB, Kafka, etc.
They started on their own Redis fork a while ago as ElastiCache has had baked in SSL for over a year and half now. While it's possible they've added it via an stunnel type software stop it, I'd bet for efficiency and simplicity it's baked into their internal fork: https://aws.amazon.com/blogs/security/amazon-elasticache-now...
They started on their own Redis fork a while ago as ElastiCache has had baked in SSL for over a year and half now. While it's possible they've added it via an stunnel type software stop it, I'd bet for efficiency and simplicity it's baked into their internal fork: https://aws.amazon.com/blogs/security/amazon-elasticache-now...
[update - re redis ssl] pretty sure thats what this upstream pull request to redis is https://github.com/antirez/redis/pull/4855
They already did their MongoDB-compatible NoSQL document database. Not sure what's under the hood but I assume an altered MongoDB engine adapted to their scalable cloud storage architecture.
https://aws.amazon.com/documentdb/
https://aws.amazon.com/documentdb/
It’s believed to be the same storage engine as their relational data offerings, aurora.
Its an emulation of MongoDB and not a very good one at that.
Oh man Amazon is going to bury Elastic (the company) atleast.
This is a very surprising move from AWS. In the past they haven't seemed to be willing to contribute to the OSS that they pick up and host. Even though they claim they'll contribute changes upstream, I doubt that Elastic would accept changes that are competitive with their commercial offerings. So you effectively get a fork.
I think this is a win for the Elastic community as a whole, but presents a real problem for Elastic the company. And that begs the question of what happens to the Elastic community if there's a real fork. And what happens if Elastic the company is very negatively impacted? Do we see a fracture of the community?
Looking even farther out, at this point does it make sense for any startup looking to create infrastructure software to open source it? If their project becomes successful, they'll get eaten up by the hosting providers. It makes smaller scale open source more commercially viable because you won't attract the attention of the providers that would come in a take your business.
Will be interesting to at least watch what early stage VC investment in open source companies looks like over the next 12-24 months. Will the open source pitch work for series A investments or will investors shy away?
I think this is a win for the Elastic community as a whole, but presents a real problem for Elastic the company. And that begs the question of what happens to the Elastic community if there's a real fork. And what happens if Elastic the company is very negatively impacted? Do we see a fracture of the community?
Looking even farther out, at this point does it make sense for any startup looking to create infrastructure software to open source it? If their project becomes successful, they'll get eaten up by the hosting providers. It makes smaller scale open source more commercially viable because you won't attract the attention of the providers that would come in a take your business.
Will be interesting to at least watch what early stage VC investment in open source companies looks like over the next 12-24 months. Will the open source pitch work for series A investments or will investors shy away?
Adding another thought to this, which is that I doubt the code behind this is a direct response to Elastic's recent license changes. This is a ton of work and I'm guessing AWS has been working on the code side of this since well before last summer. By all accounts their hosted Elastic offering is wildly successful so they've probably been hard at work to add features that Elastic formerly had as closed and commercial (before the license change).
I think their open sourcing of this work is a direct response to the recent license changes. If it weren't for those, they might have just kept this work closed and in use only for the hosted product (like they do with many other projects).
I think their open sourcing of this work is a direct response to the recent license changes. If it weren't for those, they might have just kept this work closed and in use only for the hosted product (like they do with many other projects).
Can you point me to the recent license change you mentioned?
Discussion from just over a year ago: https://news.ycombinator.com/item?id=16487440
They took the closed XPack code and put it into the open source Elastic repo, but under a commercial license. It muddied the waters for anyone looking to use or contribute to the open source.
They took the closed XPack code and put it into the open source Elastic repo, but under a commercial license. It muddied the waters for anyone looking to use or contribute to the open source.
Prior to this move, the default install of the Elastic Stack was 100% open source. X-Pack had to be intentionally installed as a plugin.
Now the default install includes X-Pack, and you have to go out of your way (assuming you even realize that there is difference) to install the 100% Apache 2.0 licensed version.
Now the default install includes X-Pack, and you have to go out of your way (assuming you even realize that there is difference) to install the 100% Apache 2.0 licensed version.
> Looking even farther out, at this point does it make sense for any startup looking to create infrastructure software to open source it?
Of course it still does, just not under permissive licenses. Hopefully the industry will converge on Kyle Mitchell's license for this, instead of everyone coming up with their own licenses: https://github.com/kemitchell/api-copyleft-license
Of course it still does, just not under permissive licenses. Hopefully the industry will converge on Kyle Mitchell's license for this, instead of everyone coming up with their own licenses: https://github.com/kemitchell/api-copyleft-license
I view that as a significant negative impact to OSS. I have a strong preference for liberal licenses like MIT or Apache 2.0. Infectious licenses like copy-left put restrictions on it that make it less appealing as open source. I wrote a bit on it here: https://www.influxdata.com/blog/copyleft-and-community-licen...
While I understand your argument, I dispute that copy-left makes things inherently less appealing as open source, or that you can make any objective claim that liberal licenses ultimately result in broader utility.
1. I personally find copy-left inherently more appealing (as something to potentially contribute to) purely because of my ideological leanings.
2. While the first order calculation of use for liberally licensed software seems clearly in favor of them, it's impossible to ever know what benefits the copy-left restrictions would bring one or more orders removed from the original distribution.
> software that you build with it, other than applications.
I'm no lawyer, but this sounds like complete nonsense.
I'm no lawyer, but this sounds like complete nonsense.
Isn't it similar to LAGPL? And EPL (but w/o the ASP loophole addressed)?
> This is a very surprising move from AWS.
I wonder why you say that? It's kind of obvious that somebody was going to do it sooner or later to protect themselves, no? Perhaps fearing the license change?
> So you effectively get a fork.
Yup, I assume that will happen, too.
I wonder why you say that? It's kind of obvious that somebody was going to do it sooner or later to protect themselves, no? Perhaps fearing the license change?
> So you effectively get a fork.
Yup, I assume that will happen, too.
Oh I assumed that someone would do it. I just didn't expect it to be AWS. Elastic is popular enough and the core open source license permissive enough that I expected some sort of fork eventually. The license of the original project is an important factor in this. For example, I wouldn't expect a fork of MongoDB because of the limitations of the AGPL license make it less appealing. With that case I'd expect what AWS already did, which was to create an API compatible system. Also notable is that they're not open sourcing that.
With Elastic, this looks like it's half marketing on AWS' part. They want to start repairing their image in the OSS world so they say they're doing this for the community.
With Elastic, this looks like it's half marketing on AWS' part. They want to start repairing their image in the OSS world so they say they're doing this for the community.
Oh, interesting. I see this as a pure attack/defense move, not just marketing themselves in the OSS world. We'll see :)
Well, they did also do this: https://aws.amazon.com/corretto/
So it does seem to be a pattern this year.
So it does seem to be a pattern this year.
Yeah. I think the motivation there was the same. Judging by https://github.com/corretto/corretto-8/graphs/contributors and https://github.com/corretto/corretto-11/graphs/contributors there is some adoption... Not a lot yet.
[deleted]
> This is a very surprising move from AWS. In the past they haven't seemed to be willing to contribute to the OSS that they pick up and host
I’m actually suprised they’re moving so slowly with OSS. I assumed they were going to go a lot more aggressively in this direction when they hired Adrian Cockcroft over 2 years ago now.
I’m actually suprised they’re moving so slowly with OSS. I assumed they were going to go a lot more aggressively in this direction when they hired Adrian Cockcroft over 2 years ago now.
> It makes smaller scale open source more commercially viable because you won't attract the attention of the providers that would come in a take your business.
Small scale open source isn't technologically viable.
> In the past they haven't seemed to be willing to contribute to the OSS
They are extremely hostile to the idea.
Small scale open source isn't technologically viable.
> In the past they haven't seemed to be willing to contribute to the OSS
They are extremely hostile to the idea.
How is small scale open source not technologically viable ? There are tons of small open source companies that build great niche open source products which are technologically competitive. I run such a tech/business (XWiki) for 15 years now. They just are not known and don't interest VCs unless you are ready to start going open core. However it is possible to be sustainable, just don't expect to be startup-rich.
I'm not going to use FOSS that doesn't have a strong community around, at least not for anything major.
> Unfortunately, we are seeing other examples where open source maintainers are muddying the waters between the open source community and the proprietary code they create to monetize the open source.
As I understand it, Amazon took ElasticSearch and monetized it, therefore competing directly with Elastic (the company that develops ElasticSearch). Elastic felt Amazon was taking advantage of Elastic's commitment to open source, and started developing new features under a proprietary license, but releasing them for free. The idea was to prevent their work from benefiting a competitor.
Basically in this blog post, Amazon is trying to make the case that they're a Good Tech Company and Elastic are the Bad Guys or something. Generally I'm all for nuance and "the truth is probably somewhere in the middle", but Elastic is the rare company that truly is committed to open source (at least at the current moment in history) and Amazon is, well, Amazon.
As I understand it, Amazon took ElasticSearch and monetized it, therefore competing directly with Elastic (the company that develops ElasticSearch). Elastic felt Amazon was taking advantage of Elastic's commitment to open source, and started developing new features under a proprietary license, but releasing them for free. The idea was to prevent their work from benefiting a competitor.
Basically in this blog post, Amazon is trying to make the case that they're a Good Tech Company and Elastic are the Bad Guys or something. Generally I'm all for nuance and "the truth is probably somewhere in the middle", but Elastic is the rare company that truly is committed to open source (at least at the current moment in history) and Amazon is, well, Amazon.
You don't understand at all. Elastic starting monetizing it, not Amazon. They raised over $100M in venture capital. [1] They had a $2.5B valuation IPO. They made many features available through subscription only for over 5 years. [2] This is in contrast to the Apache Solr project, where the updates to the engine are made to the open source project and it is released in concert with Lucene.
[1] https://techcrunch.com/2014/06/05/elasticsearch-scores-70m-i...
[2] https://www.elastic.co/blog/shield-know-security-coming-soon
[1] https://techcrunch.com/2014/06/05/elasticsearch-scores-70m-i...
[2] https://www.elastic.co/blog/shield-know-security-coming-soon
> Elastic [...] started developing new features under a proprietary license [...]. The idea was to prevent their work from benefiting a competitor. [...] Elastic is the rare company that truly is committed to open source (at least at the current moment in history) and Amazon is, well, Amazon.
I know there's more context here, and I'm sympathetic to Elastic's position, but "Elastic loves open source so much they made proprietary features to hurt their competitors!" is a bit....Orwellian as far as defenses.
I don't think this comes across as pro-Elastic as you think it does. :)
I know there's more context here, and I'm sympathetic to Elastic's position, but "Elastic loves open source so much they made proprietary features to hurt their competitors!" is a bit....Orwellian as far as defenses.
I don't think this comes across as pro-Elastic as you think it does. :)
> Elastic loves open source so much they made proprietary features to hurt their competitors!
I don't think that's a fair interpretation. They're not going on the attack, after all. Amazon's capitalization on ElasticSearch constitutes an existential threat to Elastic, not the other way around. Elastic is very much defending its existence, which, by the way, is the primary sponsor for the open source project. Moreover, the interpretation ignores that the "proprietary features" in question (including their source code) are available for free to the public.
I'm open to the idea that there were better (i.e., friendlier to open source) recourses available, but I'm not pursuaded by purist arguments that don't address the existential threat to Elastic and the ElasticSearch project.
I don't think that's a fair interpretation. They're not going on the attack, after all. Amazon's capitalization on ElasticSearch constitutes an existential threat to Elastic, not the other way around. Elastic is very much defending its existence, which, by the way, is the primary sponsor for the open source project. Moreover, the interpretation ignores that the "proprietary features" in question (including their source code) are available for free to the public.
I'm open to the idea that there were better (i.e., friendlier to open source) recourses available, but I'm not pursuaded by purist arguments that don't address the existential threat to Elastic and the ElasticSearch project.
The challenge here is that there is an implied expectation that if you started or are the primary sponsor of the project, you should be the primary recipient of revenue resulting from same. Often this expectation is paired with a belief that the project and the product are the same thing (lack of competitive differentiation).
The first is not a given at all, and the second is what leads to it being a foregone conclusion with the question being not if but when someone will eat your lunch whether its Amazon or users choosing to DIY.
The first is not a given at all, and the second is what leads to it being a foregone conclusion with the question being not if but when someone will eat your lunch whether its Amazon or users choosing to DIY.
I don't think it's a challenge at all. It's not my position, nor does my position hinge on it. Elastic is merely an important contributor to ElasticSearch, but they should not be regarded as "hostile toward open source" (esp not moreso than Amazon) because--in existential self-defense--they began writing nominally proprietary features (which they give away for free to everyone except their competitors).
Most of Elastic's critics in this thread seem to believe that, as the primary sponsor of the project, everything Elastic does must be open source (apparently) at all costs or they are somehow anti-open-source or otherwise "worse for open source" than Amazon.
Most of Elastic's critics in this thread seem to believe that, as the primary sponsor of the project, everything Elastic does must be open source (apparently) at all costs or they are somehow anti-open-source or otherwise "worse for open source" than Amazon.
releasing "new features under a proprietary license" might be a smart business move, but let's not pretend that it's pro open source.
Afaict, Elastic views Amazon's competition as an existential threat, and is releasing under a proprietary license as a last resort. Mind you, the source code is open and they're not charging for it. I'm not sure what you expect a company to do? Fall on their sword out of a dogmatic sense of Open Source Honor?
That's a straw man though- just because the action makes sense doesn't mean it supports open source software.
No, that's a straw man :). My claim was not "just because the action makes sense means it supports open source software", it was "Elastic on balance is one of the most pro-open-source companies, and recent events should be interpreted in the context of the existential threat posed by Amazon using Elastic's open source contributions to compete with Elastic".
Given that and the fact that Elastic is the primary sponsor of the ElasticSearch open source project, it is obvious that this action actually is in the best interest of open source, as it allows Elastic to continue sponsoring ElasticSearch.
Most of the criticism of Elastic in this thread seems to be predicated on the purist notion that proprietary software is the antithesis of open source software. That's obviously untrue, and we really need to disillusion ourselves of this.
Given that and the fact that Elastic is the primary sponsor of the ElasticSearch open source project, it is obvious that this action actually is in the best interest of open source, as it allows Elastic to continue sponsoring ElasticSearch.
Most of the criticism of Elastic in this thread seems to be predicated on the purist notion that proprietary software is the antithesis of open source software. That's obviously untrue, and we really need to disillusion ourselves of this.
It is not obvious that this action is actually in the best interest of open source. It's obvious that it's in the best interest of Elastic, but there is nothing intrinsically special about them that means they are the best stewards for the code until the end of time itself versus Amazon or anyone else for that matter.
The ability to fork in open source is a feature not a bug and ultimately the community of users and developers of these projects will decide what works best for them. In this case it isn't even a fork, it's new plugins that the community can consume under a FOSS license.
The ability to fork in open source is a feature not a bug and ultimately the community of users and developers of these projects will decide what works best for them. In this case it isn't even a fork, it's new plugins that the community can consume under a FOSS license.
Elastic's business model is entirely predicated on a symbiotic relationship with the open source project. Amazon doesn't have any business incentive to be the primary sponsor of the open source project; certainly not in the long term. Seems pretty obvious to me.
At that point it becomes about the nuance of defining what is really best for open source. Is it long term maintenance of ElasticSearch as it exists today, in which case I agree Elastic has the best incentive to keep it going, or is it net new open source code in which case Amazon is bringing new plugins to the table. I suspect in practice this is more of a spectrum than an obvious either/or.
I expect neither party to do anything different than they did.
Exactly, they can both be right. In Elastic's case though being right still doesn't mean they exist in 5 years. This doesn't reflect that what they did here was wrong but that by the time this started unfolding they were already in a bad position.
I don't know if both or either of them are "right". Maybe neither one is "right", they can both be "wrong" too. I'm not sure how we determine what is "right" or "wrong" here. But you asked what we expected them to do.
If by "what do you expect a company to do" you mean what is _ethical_, then what's good for their business seems like an entirely orthogonal concern to me.
If the question is: What is good for the continued viability of open source -- I honestly do not know.
If by "what do you expect a company to do" you mean what is _ethical_, then what's good for their business seems like an entirely orthogonal concern to me.
If the question is: What is good for the continued viability of open source -- I honestly do not know.
I didn't pose the original question, but in my mind "right" here is simply, made the best decision available to them at the point in time.
Best decision for what desired outcomes? Their own profit? Sure.
While I might hope for more I don't expect a company to make decision any other way.
Sure, but ya don't got to it call it "right"!
> [Elastic] started developing new features under a proprietary license
> Elastic is the rare company that truly is committed to open source
On their face, those two statements seem mutually exclusive. What context am I missing that would make them not so?
> Elastic is the rare company that truly is committed to open source
On their face, those two statements seem mutually exclusive. What context am I missing that would make them not so?
Elastic, it seems to me, is the not-at-all rare company who is truly committed to using “open source” as a marketing gimmick and tool to recruit free labor, and like most such companies resents being reduced to itself playing the role it envisions for FOSS community members by a bigger fish.
> Elastic, it seems to me, is the not-at-all rare company who is truly committed to using “open source” as a marketing gimmick
Don't think it was that nefarious. More likely that the original developers were committed to open source, and then took lots of venture capital money to keep coding and making cool things. Now those VCs want a return on their investment, and effectively have control over the product.
Don't think it was that nefarious. More likely that the original developers were committed to open source, and then took lots of venture capital money to keep coding and making cool things. Now those VCs want a return on their investment, and effectively have control over the product.
Not so much for free labor but to drive adoption. Lots of people use ES for free. When that use starts to scale, most folks end up needing to pay ES for enterprise critical features or support.
It’s a strategy that isn’t working out for a bunch of companies.
It’s a strategy that isn’t working out for a bunch of companies.
Agreed. The parent's comment's last line contradicts most of the argument.
Releasing it for 'free' is the not the same as an equitable Open Source license.
Releasing it for 'free' is the not the same as an equitable Open Source license.
It's open source meets the real world. We aren't all hippies who can sit around at MIT.
> It's open source meets the real world
It's nothing like open source; it's proprietary software that attempts to leverage the goodwill of the “open source” label without adhering to what that label means.
> We aren't all hippies who can sit around at MIT.
Look, if open source doesn't work for your business situation, that's fine, just don't lie about being committed to open source where you are truly only committed to your own profits, all while making disparaging portrayals of the people who actually are committed to open source.
It's nothing like open source; it's proprietary software that attempts to leverage the goodwill of the “open source” label without adhering to what that label means.
> We aren't all hippies who can sit around at MIT.
Look, if open source doesn't work for your business situation, that's fine, just don't lie about being committed to open source where you are truly only committed to your own profits, all while making disparaging portrayals of the people who actually are committed to open source.
As I understand it, they've released all of their code for a long time under an open source license and continue to release their source code and allow non-competitors to use it for free. All contributions from the community are also available under open source licenses? (this is my understanding, anyway)
In your mind, how should a "committed to open source" company respond to an existential threat (not only to the business, but to the project given the business is the primary sponsor of the project)? Are there better open source licenses that allow them to meet the same ends? Or should they just cease to exist, lay off their staff, and abandon the project out of a Stallmanic dream of open source?
In your mind, how should a "committed to open source" company respond to an existential threat (not only to the business, but to the project given the business is the primary sponsor of the project)? Are there better open source licenses that allow them to meet the same ends? Or should they just cease to exist, lay off their staff, and abandon the project out of a Stallmanic dream of open source?
I think they could go with a copyleft license. Or they should just announce that they are no longer committed to open source.
> Or they should just announce that they are no longer committed to open source.
Why? Because < 100% of their code is open source? This seems infeasibly idealistic, and a false dichotomy. You can be a proprietary software company that is still very committed to open source. Anyway, if < 100% is the standard, surely Amazon is worse yet, no?
Why? Because < 100% of their code is open source? This seems infeasibly idealistic, and a false dichotomy. You can be a proprietary software company that is still very committed to open source. Anyway, if < 100% is the standard, surely Amazon is worse yet, no?
> seem mutually exclusive
Open source doesn't necessarily mean free to use.
You can view entire elastic search code on github, nothing is hidden from the users.
This way of licensing might be the only way an open source company can be truly profitable.
They still make it available to most everyone free to use, as I understand it. They just don't want to do the legwork for their competitors.
[deleted]
and you know what? Elastic's cloud offering is shit. It's really expensive, it falls over every other week and their support is terrible.
AWS ElasticSearch is not great, but it's way better than Elastic.
AWS ElasticSearch is not great, but it's way better than Elastic.
If they wanted to monetize it, why not patent the new features, demand royalties from AWS, set royalty rates of $0 for small to mid size businesses, and keep it open source? Or customize the patent so that it's only enforceable against AWS.
I realize "patent" is a dirty word around here, but aside from that, what are the issues with the above strategy and is there truly no possible way around those issues?
I realize "patent" is a dirty word around here, but aside from that, what are the issues with the above strategy and is there truly no possible way around those issues?
I realize our patent system is totally fucked up, but hopefully not so fucked up that things like "user authentication" can be patented.
> I realize our patent system is totally fucked up, but hopefully not so fucked up that things like "user authentication" can be patented.
You kidding? You have no idea how many user authentication patent battles have been fought.
You kidding? You have no idea how many user authentication patent battles have been fought.
Have a look at patents for rodent traps. Over 4000 and counting IIRC. Get inspired.
Or not. I'm no fan of software patents.
Or not. I'm no fan of software patents.
Amazon patented 1 click checkout.
meh, amazon sues over the patents and gets them tossed out then introduces a compatible API anyway.
You can't beat Amazon. You can't beat google. You can't beat Microsoft.
You can't beat Amazon. You can't beat google. You can't beat Microsoft.
Ironically, Elastic's cloud offering I believe runs on AWS...
So not in the middle then. Amazon the bad guy here.
My personal, probably disagreeable opinion, but... The narrative that Amazon is somehow screwing over open source is getting old. I don't really think anything is 'screwing over open source', but if it's anything, it is the notion that we should severely restrict the licenses to ensure only the right people can monetize it. To me, the original spirit of open source is "I don't care what happens to this after I release it," like Linus releasing Linux so long ago. To that end, Amazon has certainly leveraged hundreds or thousands of FOSS projects over the years, but only a few have cried foul about it, and they happen to be startups releasing open source software.
I'm totally supportive of developers and their rights to release software under whatever license they feel appropriate, but if you release open source software, benefit from your software being open source, for probably years, and then someone else monetizes it... I'd argue you really ought to have seen that coming from the get-go. The license explicitly allowed it, it's not a loop hole that nobody knew about, it's absolutely intentional.
If people want to move over to shared source licenses like SSPL, or heck, even just close the source... that's fine too. But please don't try to call it open source or defending open source. It's defending a profit model that stopped working. Totally reasonable thing to do, but it's nothing to do with open source being corrupted.
I'm totally supportive of developers and their rights to release software under whatever license they feel appropriate, but if you release open source software, benefit from your software being open source, for probably years, and then someone else monetizes it... I'd argue you really ought to have seen that coming from the get-go. The license explicitly allowed it, it's not a loop hole that nobody knew about, it's absolutely intentional.
If people want to move over to shared source licenses like SSPL, or heck, even just close the source... that's fine too. But please don't try to call it open source or defending open source. It's defending a profit model that stopped working. Totally reasonable thing to do, but it's nothing to do with open source being corrupted.
I think Eben, rms, et al are 100% right when they observe that 'open source' hides the actual, important point -- even if the person you're talking to doesn't want to talk about freedom(s).
You used the phrase 'open source' 9 times in your three paragraphs, but didn't mention 'free' or 'freedom'. (Hmm, the F in FOSS may count as one veiled instance.)
Talking in terms of freedom for people to use free software however they want makes more sense than trying to dance through the forest of varyingly-free 'open source' licences.
You used the phrase 'open source' 9 times in your three paragraphs, but didn't mention 'free' or 'freedom'. (Hmm, the F in FOSS may count as one veiled instance.)
Talking in terms of freedom for people to use free software however they want makes more sense than trying to dance through the forest of varyingly-free 'open source' licences.
I used to disagree on this point, but based on the way things have gone lately I think it's worth reconsidering. I definitely use the term 'FOSS' to try to convey that I'm talking about the term open source to mean free software. The trouble is, I don't necessarily want to be looped in with all of the opinions of rms, for example. rms has a different idea of what 'free software' entails, as evidenced by GPLv3. (Personally I still vastly consider GPLv3 to be an acceptable open source license, but you can see Linus Torvalds disagrees pretty staunchly with its added restrictions.)
> rms has a different idea of what 'free software' entails, as evidenced by GPLv3
I think we have to separate definitions from strategy/goals: the OSI open source definition and he FSF Free Software definition—the latter of which seems to be what RMS views Free Software as entailing—a remarkably similar. The difference seems to be that RMS and the FSF view preventing non-Free software as an important goal, and this prefer licenses which tend to have the effects of preventing downstream non-Free derivatives and encouraging people to relicense other software under them, whileany others in the FOSS universe prefer promoting FOSS creation and use to preventing non-FOSS software..
I think we have to separate definitions from strategy/goals: the OSI open source definition and he FSF Free Software definition—the latter of which seems to be what RMS views Free Software as entailing—a remarkably similar. The difference seems to be that RMS and the FSF view preventing non-Free software as an important goal, and this prefer licenses which tend to have the effects of preventing downstream non-Free derivatives and encouraging people to relicense other software under them, whileany others in the FOSS universe prefer promoting FOSS creation and use to preventing non-FOSS software..
May be I am getting the wrong ends on both comment.
But I think he described it very well. Open Source, I am opening it and I don't care about Your Freedom, nor my Freedom or anyone's Freedom. And you should not judge me whether I care about it being Free or Not.
But I think he described it very well. Open Source, I am opening it and I don't care about Your Freedom, nor my Freedom or anyone's Freedom. And you should not judge me whether I care about it being Free or Not.
> ... I am opening it ...
This is the crux.
No two people agree on what this actually means, and it sidesteps any discussion of freedoms.
This is the crux.
No two people agree on what this actually means, and it sidesteps any discussion of freedoms.
If we think of open source as an ideology that drove people to plant trees and tend them so that users get fruits for free at a time when fruits are only commercially available then the point becomes a bit more complicated.
We now have a cottage industry of people who harvest the fruits and sell them so end users don't have to pick them themselves and also others who use the free fruits to manufacture and sell milkshakes at higher margins than if they would have to grow their own fruits or buy them.
The relationship with end users and ideology is broken as is the pipeline of new contributors as end users do not anymore see the value and ideas that drove the initial plantations as they deal with fruit pickers and milkshake makers.
At this point open source loses its reason to exist and the people still planting and offering fruits for free will inevitably question what they are doing if they are only enabling third party businesses, but inertia means they will continue as that's their life work. But there is unlikely to be a second generation after them so that's the end of the movement. The story here is the commercial interests have cut the branch on which they and everyone else sit. For them it doesn't matter its just money not ideology and they will find ways to profit but the loss is to end users. So for open source to be meaningful its has to think carefully about how it will interact with commercial interests as mix and match will kill it in a generation.
We now have a cottage industry of people who harvest the fruits and sell them so end users don't have to pick them themselves and also others who use the free fruits to manufacture and sell milkshakes at higher margins than if they would have to grow their own fruits or buy them.
The relationship with end users and ideology is broken as is the pipeline of new contributors as end users do not anymore see the value and ideas that drove the initial plantations as they deal with fruit pickers and milkshake makers.
At this point open source loses its reason to exist and the people still planting and offering fruits for free will inevitably question what they are doing if they are only enabling third party businesses, but inertia means they will continue as that's their life work. But there is unlikely to be a second generation after them so that's the end of the movement. The story here is the commercial interests have cut the branch on which they and everyone else sit. For them it doesn't matter its just money not ideology and they will find ways to profit but the loss is to end users. So for open source to be meaningful its has to think carefully about how it will interact with commercial interests as mix and match will kill it in a generation.
There are free milkshake plants, it's the distros. The distributors take all of the open source projects, combine them, root out the bad fruits (DFSG non-compliant, etc.), and offer it to everyone for free. I don't have to pick my fruit from the AOSP tree and build my own image. I can just go to Lineage OS and get prebuilt images for a multitude of phones. These projects of course only reach a minority of people: Lineage OS has about 2 million users, while the entire Android market has > 2 billions. But they exist. The free milkshakes set a minimum bar that every milkshake plant has to meet.
Another thing to consider is the aspect of tree tending. If you are a professional tree planter who has worked at a tree plantation with a big wall around it, once you left, you won't be able to point at the tree any more and say "look, I planted it". E.g. when trying to get hired at another tree plantation. The only thing left of your own creation would be the diminishing memories you had of the tree growing, having its first leaves, first real bark, etc. Maybe your company won't like your tree for some reason and burn it. If it's outside, it can't be logged. If the company pays you to tend the tree, if they fire you you will still be able to tend it at a different company. Red Hat or Sun can be bought but the trees were outside of the walls, so the damages to the community were comparatively minor.
Another thing to consider is the aspect of tree tending. If you are a professional tree planter who has worked at a tree plantation with a big wall around it, once you left, you won't be able to point at the tree any more and say "look, I planted it". E.g. when trying to get hired at another tree plantation. The only thing left of your own creation would be the diminishing memories you had of the tree growing, having its first leaves, first real bark, etc. Maybe your company won't like your tree for some reason and burn it. If it's outside, it can't be logged. If the company pays you to tend the tree, if they fire you you will still be able to tend it at a different company. Red Hat or Sun can be bought but the trees were outside of the walls, so the damages to the community were comparatively minor.
I hope nobody takes strong offense to this, but I don't really care for the hippie-esque philosophical reasoning behind open source. My own open source contributions are probably weak compared to many here, but I've been at least involved in a fair few open source projects since quite a young age. Bottom line: I don't use open source because it makes me feel good, I use it because it's better.
Linux is the canonical example. It gets contributions from everyone. Why? Well, I'm no expert, but it looks like it's because everyone wants to benefit from the latest changes, and the easiest way to do that is if everyone commits. Diverging forks are expensive, and get less peer review.
Open Source is a weird beast inside capitalism; it's this common ground between a bunch of varied corporations and a pool of random users. The fact that this amazing piece of infrastructure is just available and constantly maintained by its own users is more amazing than any rant about software freedom I've ever read.
Best of luck to all the FOSS developers trying to put food on the table through their open source work, I really do wish them the best, but let's at least be fair; traditional open source projects are doing great. Better than ever, imo. Whether it's desktop apps like Krita and Blender, infrastructure like the Linux kernel, web frameworks, programming languages, virtual machines...
Linux is the canonical example. It gets contributions from everyone. Why? Well, I'm no expert, but it looks like it's because everyone wants to benefit from the latest changes, and the easiest way to do that is if everyone commits. Diverging forks are expensive, and get less peer review.
Open Source is a weird beast inside capitalism; it's this common ground between a bunch of varied corporations and a pool of random users. The fact that this amazing piece of infrastructure is just available and constantly maintained by its own users is more amazing than any rant about software freedom I've ever read.
Best of luck to all the FOSS developers trying to put food on the table through their open source work, I really do wish them the best, but let's at least be fair; traditional open source projects are doing great. Better than ever, imo. Whether it's desktop apps like Krita and Blender, infrastructure like the Linux kernel, web frameworks, programming languages, virtual machines...
This fruits analogy is really good!
I would like to belatedly disclose that I an an employee of Google, and that the opinions I have expressed are mine and not my employer's. I apologize for not disclosing in the parent post, most of my replies here were written in a highly off the cuff, passionate manner as this subject matter has long been very important to me personally, as a long time FOSS user and small time contributor. I did not expect it to resonate with so many people.
This aside, I also definitely hope nobody feels offended by these opinions, open source means different things to different people. Regardless of what it means, I'm hoping the best for the future of open source, and I remain strongly optimistic.
This aside, I also definitely hope nobody feels offended by these opinions, open source means different things to different people. Regardless of what it means, I'm hoping the best for the future of open source, and I remain strongly optimistic.
> "I don't care what happens to this after I release it,"
That is completely wrong, even for Linus, but especially to the ones that came before who called it free software. They did care what happened to it. They wanted it to stay free such that it continues to benefit society and enables people to do computing freely.
That is completely wrong, even for Linus, but especially to the ones that came before who called it free software. They did care what happened to it. They wanted it to stay free such that it continues to benefit society and enables people to do computing freely.
Amazon's internal policies towards open source and side projects are incredibly restrictive (in my opinion). If Amazon doesn't really want its employees to engage with open source, how genuine are the motives here?
Disclosure: I work at Amazon, and from time to time I help out with open source policy.
There have been a number of changes to Amazon's policies over the years, and personal participation ("outside activity") is really straightforward now. For the vast majority of cases you just need to submit a ticket with details, and then it is auto-approved.
For comparison, you can have a look at Google's open source policy on personal projects here: https://opensource.google.com/docs/iarc/
If you are an employee and have questions, drop me a line at msw at amazon.com
There have been a number of changes to Amazon's policies over the years, and personal participation ("outside activity") is really straightforward now. For the vast majority of cases you just need to submit a ticket with details, and then it is auto-approved.
For comparison, you can have a look at Google's open source policy on personal projects here: https://opensource.google.com/docs/iarc/
If you are an employee and have questions, drop me a line at msw at amazon.com
> and then it is auto-approved
So the prospects of Amazon reimplement all the popular open-source software someday is viable?
So the prospects of Amazon reimplement all the popular open-source software someday is viable?
Frankly it's irrelevant. Nobody ever said you have to contribute back to open source to benefit from it, though I think you'd be stupid not to. Amazon released their fork instead of keeping it in house, could be a PR move, but I believe they just want to keep the advantages Elastic had being open source.
But honestly, whether the companies are 'right' or not isn't my point, and I hope it's not what people take away from my ranting. I only wanted to make a point about the narratives around FOSS being destroyed. People chose to build businesses on open source; sometimes it worked, sometimes it didn't. That's all we're learning.
Contrary to reports of its death, open source is alive and well.
But honestly, whether the companies are 'right' or not isn't my point, and I hope it's not what people take away from my ranting. I only wanted to make a point about the narratives around FOSS being destroyed. People chose to build businesses on open source; sometimes it worked, sometimes it didn't. That's all we're learning.
Contrary to reports of its death, open source is alive and well.
modern open-sourcers don't care about software freedom. They want free contributors and marketshare so they can 'monetise' and 'exit'.
Gotta learn to spot the fakes.
Gotta learn to spot the fakes.
FOSS devs have the same bills as anyone else.
Nobody is obligated to work on FOSS unless somebody is outright paying them to.
But people that work on FOSS, as owners of the IP they create, are also allowed to put whatever restrictions they want on it; if you don't like those restrictions, don't use it. Pretty simple.
Sure. The question is - do they still get to claim the "FOSS" moral high ground and branding, once they've "put whatever restrictions they want on it".
I think a _lot_ of people say "No" to that question. Are Redis/Mongo et al "Free Open Source Software"? I tend to think not. I'm completely behind their right to change the license freedoms they choose to grant.
But at least in my opinion, it's no longer "free open source software".
It's quite clearly not rms's definition of free software. It's less free than the EFF's definition of gpl (in any of its versions). It's almost certainly not what ESR means by "Open Source".
I don't know who gets to say what is and isn't "FOSS", but I'd suggest all three of those have at the very least "prior art" ownership of the definition, and the Mongo/Redis clearly do not have any standing to define that term...
I think a _lot_ of people say "No" to that question. Are Redis/Mongo et al "Free Open Source Software"? I tend to think not. I'm completely behind their right to change the license freedoms they choose to grant.
But at least in my opinion, it's no longer "free open source software".
It's quite clearly not rms's definition of free software. It's less free than the EFF's definition of gpl (in any of its versions). It's almost certainly not what ESR means by "Open Source".
I don't know who gets to say what is and isn't "FOSS", but I'd suggest all three of those have at the very least "prior art" ownership of the definition, and the Mongo/Redis clearly do not have any standing to define that term...
I don't see why you'd think I don't support that. It's pretty much my point.
Last year Elastic opened up the source code of their commercial X-Pack offering: https://www.elastic.co/blog/doubling-down-on-open. This means that these components (security, alerting, etc.) are now available in the GitHub repository, but they are covered by the proprietary Elastic license instead of the Apache 2.0 license like the rest of the software. Some of the proprietary parts can be used for free, some need a paid commercial subscription. None of them are true open source (free as in freedom), though.
However, this is also when Elastic started muddying the waters (to quote the AWS blog post). Most of the new features across the Elastic stack that were added in the past couple of months (Index Lifecycle Management, APM UI, Infrastructure and Logs UI, Kibana multi-tenancy, Kibana Canvas) are not added to the Apache 2.0 codebase but are only available in the free-but-not-open version. These features can be used for free with the Basic subscription (no registration needed) but only under the terms of the Elastic license.
And this Elastic license is where AWS feels the pain. It clearly prohibits SaaS offerings:
> You agree not to: [..] use Elastic Software Object Code for providing time-sharing services, any software-as-a-service, service bureau services or as part of an application services provider or other service offering (collectively, "SaaS Offering") [..]
(from https://github.com/elastic/elasticsearch/blob/6.6/licenses/E...)
The license is very similar to moves that MongoDB and Redis made recently. Only Elastic is just now doubling down on adding new (and highly demanded) features to their proprietary offering.
However, this is also when Elastic started muddying the waters (to quote the AWS blog post). Most of the new features across the Elastic stack that were added in the past couple of months (Index Lifecycle Management, APM UI, Infrastructure and Logs UI, Kibana multi-tenancy, Kibana Canvas) are not added to the Apache 2.0 codebase but are only available in the free-but-not-open version. These features can be used for free with the Basic subscription (no registration needed) but only under the terms of the Elastic license.
And this Elastic license is where AWS feels the pain. It clearly prohibits SaaS offerings:
> You agree not to: [..] use Elastic Software Object Code for providing time-sharing services, any software-as-a-service, service bureau services or as part of an application services provider or other service offering (collectively, "SaaS Offering") [..]
(from https://github.com/elastic/elasticsearch/blob/6.6/licenses/E...)
The license is very similar to moves that MongoDB and Redis made recently. Only Elastic is just now doubling down on adding new (and highly demanded) features to their proprietary offering.
If you read the Elastic License, it's actually worse than Mongo & Redis -- you can't modify it and use it for anything other than testing. You can basically install & use it only (definition of License, then the restrictions sections); if your testing uncovers something wrong... what are you supposed to do, under the terms of their license?
Discover Lucene.
Comparing Lucene to Elasticsearch is very apples to oranges, in my opinion. It's like if MySQL did something people didn't like and the answer was "Discover InnoDB".
Sure, Lucene might be all you need, if all you need is some basic freetext search on a small set of data that fits on a single machine.
Though maybe you misspoke and meant to say "Discover Solr"...
Sure, Lucene might be all you need, if all you need is some basic freetext search on a small set of data that fits on a single machine.
Though maybe you misspoke and meant to say "Discover Solr"...
Hm... Nope, ElasticSearch is a good entry point into Lucene. As your software needs grow, you may discover the benefits of going a level below. Start with ELK, then discover ElasticSearch, then use Lucene with your own way of distributing data over multiple machines if needed.
Personally, I never had to use more than 1 machine even for huge data sets (JFYI, Orbis and Wikipedia fit onto one bare metal server).
Personally, I never had to use more than 1 machine even for huge data sets (JFYI, Orbis and Wikipedia fit onto one bare metal server).
Lucene's pretty great. The segments file model even works pretty well with S3.
Elastic has not and probably wouldn't open the Security plugin because it _was_ the sole reason to buy X-Pack, one of main ones as of now. They've opened only monitoring and reporting because there are other ways to do it without X-Pack anyway, all other parts in OSS code just have interfaces so anyone's plugins could interact with them.
They have open sourced it (https://github.com/elastic/elasticsearch/tree/master/x-pack/...), but you cannot (legally) use it without a paid subscription. Same goes for PDF and PNG reports in Kibana.
Monitoring is available in the free subscription, which cannot be used by SaaS providers. And now that we have an alternative: the Elastic monitoring plugin is pretty neat, but the Open Distro Performance Analyzer looks way more powerful. I’ve had the Elastic monitoring UI and a regular ‘perf top’ open with still no clue as to why Elasticsearch was eating CPU (relative to the query load). It looks like Performance Analyzer brings far more to the table.
Monitoring is available in the free subscription, which cannot be used by SaaS providers. And now that we have an alternative: the Elastic monitoring plugin is pretty neat, but the Open Distro Performance Analyzer looks way more powerful. I’ve had the Elastic monitoring UI and a regular ‘perf top’ open with still no clue as to why Elasticsearch was eating CPU (relative to the query load). It looks like Performance Analyzer brings far more to the table.
> They have open sourced it (https://github.com/elastic/elasticsearch/tree/master/x-pack/...), but you cannot (legally) use it without a paid subscription
That's called Source-available and sometimes that's even worse because you (legally) cannot re-implement something similar after you have looked at the code.
That's called Source-available and sometimes that's even worse because you (legally) cannot re-implement something similar after you have looked at the code.
[deleted]
I've missed it.
> but you cannot (legally) use it without a paid subscription
Sounds worse than my thoughts in the comment, considering the pricing for self-hosted nodes (happend that I know them).
> but you cannot (legally) use it without a paid subscription
Sounds worse than my thoughts in the comment, considering the pricing for self-hosted nodes (happend that I know them).
For improved context, read the HN thread around that announcement:
https://news.ycombinator.com/item?id=16487440
https://news.ycombinator.com/item?id=16487440
Thanks for the background.
This is an interesting new "threat" to the "open core" business model. ES makes proprietary extensions to support their FOSS core product - then a tech behemoth clones these features for their own hosted service, but makes them Free as well. Good for the consumer, but bad for the company that originally created the core FOSS technology and best for Amazon.
Have there been any examples of this happening before?
Have there been any examples of this happening before?
I mean, this is basically what happened to commercial UNIXes, is it not? Tech behemoths decided it was more in their interests to fund the development of a free-software UNIX clone (the GNU/Linux ecosystem) than to keep paying Sun et al. The UNIX wars were, I'm sure, very meaningful for the participants, but ten years later it turned out there was more money to be made in building things on top of the OS than building the OS.
Hell, even before Linux was popular, gcc took off in the commercial Unix space because a) it didn't cost anything, while the official Sun compiler was ludicrously expensive, and b) it was more performant than the official Sun compiler.
The whole MongoDB thing recently.
And it doesn't appear that AWS is hosting these right now, just distributing so they must be paying some kind of licensing for ES. But I'm sure that will change.
And it doesn't appear that AWS is hosting these right now, just distributing so they must be paying some kind of licensing for ES. But I'm sure that will change.
Re MongoDB, I don't think their pretending to be opensource anymore, just source visible, see
https://lukasatkinson.de/2019/mongodb-no-longer-seeks-osi-ap...
Re OpenES, its an apache licensed distro without proprietary commercial bits.
One of the concerns I have had with the recent trending of some opensource companies (elastic, timescaledb, and others), is that those who mix in proprietary code into their opensource repos, effectively taint contributors who can't even look at git history on a project without viewing stuff thats not opensource.
https://lukasatkinson.de/2019/mongodb-no-longer-seeks-osi-ap...
Re OpenES, its an apache licensed distro without proprietary commercial bits.
One of the concerns I have had with the recent trending of some opensource companies (elastic, timescaledb, and others), is that those who mix in proprietary code into their opensource repos, effectively taint contributors who can't even look at git history on a project without viewing stuff thats not opensource.
The blog post says that some of these features now appear in ESS (AWS hosted ES), and that the rest will arrive soon.
Did AWS do the same thing with Mongo?
Did AWS do the same thing with Mongo?
They created a compatible hosted database.
I believe in response to some licensing changes MongoDB made, so the AWS service targets the last permitted license release by Mongo.
https://aws.amazon.com/blogs/aws/new-amazon-documentdb-with-...
*I forgot to mention, they did not release the code for this as you may have been looking for apple-to-apple comparison with the ES situation.
I believe in response to some licensing changes MongoDB made, so the AWS service targets the last permitted license release by Mongo.
https://aws.amazon.com/blogs/aws/new-amazon-documentdb-with-...
*I forgot to mention, they did not release the code for this as you may have been looking for apple-to-apple comparison with the ES situation.
Hi, I work at AWS on Compute services, but not directly on database services. I also work on broader open source topics at Amazon.
Amazon DocumentDB was in development for a long time, and development started well before any of the license changes that MongoDB (the company) made to MongoDB (the database).
Amazon DocumentDB was in development for a long time, and development started well before any of the license changes that MongoDB (the company) made to MongoDB (the database).
> Good for the consumer, but bad for the company that originally created the core FOSS technology and best for Amazon.
I don't know if this is a bad pattern in general for the company that originally created the core FOSS technology. If your business model is support/consultancy this may work to your advantage.
I don't know if this is a bad pattern in general for the company that originally created the core FOSS technology. If your business model is support/consultancy this may work to your advantage.
There's a huge legal precedence loophole in the GPL, where one can essentially white label a piece of software and resell it as their own product.
For example, I could build a multi-site WordPress instance, white label it, and resell it as a custom blogging platform.
For example, I could build a multi-site WordPress instance, white label it, and resell it as a custom blogging platform.
That's not a loophole, that's pretty much a design choice in the license.
Definitely creates a new risk model for hybrid OSS-Enterprise software: build something attractive enough for the big players to co-opt and they might just, ah, fork you up. Now Elastic either accepts pull requests that create clean OSS versions of their pay-locked functionality, or they accept the existence of a feature-plus “Amazon Elasticsearch” fork in the market. While I give Cockcroft all the credit in the world (remembering him from his Sun days), this is still a tough spot for Elastic to be in. (More discussion in the Twitters: https://mobile.twitter.com/_msw_/status/1105260461149151232)
From that twitter feed
"adrian cockcroft:
What should we have done instead, that Elastic would have agreed to? We have customers to support"
And this is where the spot he is in gets tighter. Because now the aws business model threats anybody using X as Amazon's user that should be monetized.
So I don't see Amazon backing off profits, and I don't see the OSS community getting any less pissed off from not seeing a piece of the cake.
This story (Mongo, elastic co, etc) seems far from being over.
"adrian cockcroft:
What should we have done instead, that Elastic would have agreed to? We have customers to support"
And this is where the spot he is in gets tighter. Because now the aws business model threats anybody using X as Amazon's user that should be monetized.
So I don't see Amazon backing off profits, and I don't see the OSS community getting any less pissed off from not seeing a piece of the cake.
This story (Mongo, elastic co, etc) seems far from being over.
Amazon should pay money to the people who create the open source. Not necessarily because a license compels them to, but because it's the right thing to do. If they don't, then the ecosystem collapses.
A good example is RedHat. There is basically no money in simply selling operating system licenses anymore, but someone has to keep developing it. Amazon makes lots of money off of EC2 instances. They could easily throw a few million a year into paying kernel developers. But will they? Are they just going to be a free rider, while keeping anyone else from being able to make any money too?
A good example is RedHat. There is basically no money in simply selling operating system licenses anymore, but someone has to keep developing it. Amazon makes lots of money off of EC2 instances. They could easily throw a few million a year into paying kernel developers. But will they? Are they just going to be a free rider, while keeping anyone else from being able to make any money too?
>Amazon should pay money to the people who create the open source. Not necessarily because a license compels them to, but because it's the right thing to do. If they don't, then the ecosystem collapses.
Elasticsearch is built on top of Lucene. When Elastic received triple digit millions in VC funding, did they pay the Lucene developers? They're not even Apache Foundation sponsors, as best as I can tell.
>Amazon makes lots of money off of EC2 instances. They could easily throw a few million a year into paying kernel developers. But will they?
Amazon contributes back to the Linux kernel. How much contribution is needed to make it "right"? How do we determine it? Does Amazon need to employ them directly, or sponsor projects? What about their Platinum sponsorship of the Apache Foundation?
Elasticsearch is built on top of Lucene. When Elastic received triple digit millions in VC funding, did they pay the Lucene developers? They're not even Apache Foundation sponsors, as best as I can tell.
>Amazon makes lots of money off of EC2 instances. They could easily throw a few million a year into paying kernel developers. But will they?
Amazon contributes back to the Linux kernel. How much contribution is needed to make it "right"? How do we determine it? Does Amazon need to employ them directly, or sponsor projects? What about their Platinum sponsorship of the Apache Foundation?
Amazon do pay people to contribute to the kernel, and they have their own Linux distribution which when you poke it looks an awful lot like RHEL/CentOS but different due to some things Amazon add/change, so I'm not sure that's the greatest example. Red Hat probably aren't thrilled about that situation, but in the same breath are a larger company and not under the same pressure to simply survive at this point.
> Definitely creates a new risk model for hybrid OSS-Enterprise software: build something attractive enough for the big players to co-opt and they might just, ah, fork you up.
That's definitely not new; that has been observed as a natural consequence of Free Softw are licensing since at least 1990-ish. And I only say that as the latest possible date because it's when I first got internet access and saw discussion of it.
That's definitely not new; that has been observed as a natural consequence of Free Softw are licensing since at least 1990-ish. And I only say that as the latest possible date because it's when I first got internet access and saw discussion of it.
As an active developer and maintainer of a cluster, I strongly disapprove of this move by AWS.
By forking on an earlier instance of the codebase for ES and Kibana they're not only creating an open source version of the code but also attempting a fork of the community - plugin developers, user groups, etc. It's extremely frustrating. EDIT: If folks actively building plugins have felt pain here because of decisions by Elastic, then my view on this point changes considerably, obviously.
I find it very hard to believe there was no compromise to be had here between the current Elastic roadmap for their product and their previous work.
How is this good for users of the current tools? How does this help create better capabilities on a common platform? Will this encourage users to build businesses on elasticsearch open source elements?
I'm not sure where the community ends up after this, so I'm not sure I can support by using AWS' elasticsearch tools.
By forking on an earlier instance of the codebase for ES and Kibana they're not only creating an open source version of the code but also attempting a fork of the community - plugin developers, user groups, etc. It's extremely frustrating. EDIT: If folks actively building plugins have felt pain here because of decisions by Elastic, then my view on this point changes considerably, obviously.
I find it very hard to believe there was no compromise to be had here between the current Elastic roadmap for their product and their previous work.
How is this good for users of the current tools? How does this help create better capabilities on a common platform? Will this encourage users to build businesses on elasticsearch open source elements?
I'm not sure where the community ends up after this, so I'm not sure I can support by using AWS' elasticsearch tools.
As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet).
Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert
Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql
Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard
All of them have Apache 2.0 license.
Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert
Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql
Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard
All of them have Apache 2.0 license.
> why wasn't ElasticSearch SQL used / built upon for SQL?
According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin."
Sounds like they are building on it, as you suggested.
[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....
According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin."
Sounds like they are building on it, as you suggested.
[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....
> Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard
Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl...
floragunn GmbH Copyright 2015-2018 floragunn GmbH
Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl...
floragunn GmbH Copyright 2015-2018 floragunn GmbH
For SearchGuard, only the basic auth is Apache 2.0. Anything practical you need to pay to use it.
From a look at the source code, Amazon has Apached 2.0 the entire Floragunn Searchguard codebase (including the commercial part)
Why isn't AWS paying to use it, then? It's not like they don't have the money?
It's not just about AWS, other people need to use this as well. Why are security features a paid for addition? Why not make core features a paid for addition instead? I've always hated Elastic (the company) for this decision, please stop trying to give away free things that are not secure, and then charge for security.
I'd recommend browsing the code to find out more about the security module:
https://github.com/opendistro-for-elasticsearch/security/blo...
https://github.com/opendistro-for-elasticsearch/security/blo...
there's no "they", there's only us. AWS paying to use it means we are paying to use it. Between that and a truly Apache 2.0 project, I think it's a no-brainer for everyone.
Why shouldn't you pay to use it, then?
More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?
More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?
I'm very glad that they've done this.
The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc.
Imagine if Linus said that the next version of Linux would be paid-only.
Within 15 minutes there'd be a new librelinux repo that people could contribute to instead.
Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.
The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc.
Imagine if Linus said that the next version of Linux would be paid-only.
Within 15 minutes there'd be a new librelinux repo that people could contribute to instead.
Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.
If ES didn't do the weird license change thing, we wouldn't be here. Either Amazon wouldn't have used it, or they'd be paying. Including non-com features in the default install was not a good way to play things.
Would you feel the same if AWS built a drop in/api compatible system from scratch?
Would you feel the same if AWS built a drop in/api compatible system from scratch?
Beyond the thoughtful ideas below, an official blog post would be cool for that. Specifically, Elastalert and ODfE Alerting are completely different.
From the pom.xml of Open Distro for ElasticSearch: the fork is from elasticsearch-6.5.4, which is the latest stable release. Only alternative would have been v6.6.1. EDIT: also, nothing is there to believe they would not bump to higher versions in later versions.
6.6.1 is the latest stable release. 6.5.4 was released 19 december 2018 [1], so they’re not too far behind, though.
[1] https://www.elastic.co/blog/elastic-stack-6-5-4-released
[1] https://www.elastic.co/blog/elastic-stack-6-5-4-released
Thanks for this clarification!
From the related AWS blog (so at least a few grains of salt):
>>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path.
... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.
>>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path.
... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.
There is certainty around the open-source bits and the license - this is true. That said elastic the company has also seems to have brought the product a long way after becoming a company. Having a dedicated staff of engineers and developers behind a thing is helpful for that, and they should be able to make some money as part of this if we want to see future open source products advance similarly. (There is a difference too between consulting / services and money “at scale” - AWS is removing an at-scale component here)
That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.
That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.
I run a company that offers consulting and hosted services around a 100% open core (https://geocode.earth, shameless plug).
As someone who has a very strong desire to continue to be able to make a living while making open source software, I've been thinking long and hard about all the possible paths to take while maintaining open-source software as a job that pays the bills enough to run at least a small business.
Some paths and possible outcomes are:
- Open-core: Make most code pure open source, and then save some good stuff for pure proprietary. The lines are not _too_ blurred here, but this can get annoying. (Elastic's X-Pack was _super_ annoying to deal with back in the 2.x days). Risks: you have to balance crippling the core versus building compelling proprietary features. You have to add additional complexity into both codebases to deal with a clean delineation. Amazon (or other bigco) can re-implement open-source replacements for your proprietary parts.
- Source-available: This in my mind includes any of the recent Redis Labs style licenses. Pros: more convenient from a setup perspective. Cons: Incredible legal risk for your users (compliance is harder). Uncertainty as all these licensees are new and possibly one-offs. Currently, there is a lack of goodwill around these licenses. Amazon can re-implement your entire interface (like with Mongo).
- All open source: Simple, clean, but Amazon can just take all your code and replace you, right?
It's possible to assume that all scenarios end poorly, but I really don't think so.
Take any famous chef. They probably have published a cookbook with recipes for many, if not all of their most popular dishes. People still eat at their restaurants for at least the following reasons:
- They want to experience the dish from the actual chef's establishment. Whether it's to be sure they're getting the real deal or simply for the image aspects of the experience, it's still worth it.
- They know that the chef is always experimenting and pushing new things that aren't _yet_ in a cookbook, and they want to try it.
Bringing this back to software, I am confident Elastic still has a reputation of being able to build a better Elasticsearch hosted service than AWS (in my experience, Elastic's is far superior). I also am confident that Elastic will continue to innovate using their proven experience building search products in the future, and that's a good reason to use their products or software over AWS.
I believe that if you want to make money in open source you should do it by having valuable experience with some particular open source software that others are willing to pay for, not by building legal barriers to others doing the same thing. That's what I plan to do for my business and I believe Elastic can and will do it too.
My hope is that in a few years after all these "clever" attempts to build moats around open source have proven futile, people will go back to good old-fashioned experience leading the way. But maybe I shouldn't hold my breath.
As someone who has a very strong desire to continue to be able to make a living while making open source software, I've been thinking long and hard about all the possible paths to take while maintaining open-source software as a job that pays the bills enough to run at least a small business.
Some paths and possible outcomes are:
- Open-core: Make most code pure open source, and then save some good stuff for pure proprietary. The lines are not _too_ blurred here, but this can get annoying. (Elastic's X-Pack was _super_ annoying to deal with back in the 2.x days). Risks: you have to balance crippling the core versus building compelling proprietary features. You have to add additional complexity into both codebases to deal with a clean delineation. Amazon (or other bigco) can re-implement open-source replacements for your proprietary parts.
- Source-available: This in my mind includes any of the recent Redis Labs style licenses. Pros: more convenient from a setup perspective. Cons: Incredible legal risk for your users (compliance is harder). Uncertainty as all these licensees are new and possibly one-offs. Currently, there is a lack of goodwill around these licenses. Amazon can re-implement your entire interface (like with Mongo).
- All open source: Simple, clean, but Amazon can just take all your code and replace you, right?
It's possible to assume that all scenarios end poorly, but I really don't think so.
Take any famous chef. They probably have published a cookbook with recipes for many, if not all of their most popular dishes. People still eat at their restaurants for at least the following reasons:
- They want to experience the dish from the actual chef's establishment. Whether it's to be sure they're getting the real deal or simply for the image aspects of the experience, it's still worth it.
- They know that the chef is always experimenting and pushing new things that aren't _yet_ in a cookbook, and they want to try it.
Bringing this back to software, I am confident Elastic still has a reputation of being able to build a better Elasticsearch hosted service than AWS (in my experience, Elastic's is far superior). I also am confident that Elastic will continue to innovate using their proven experience building search products in the future, and that's a good reason to use their products or software over AWS.
I believe that if you want to make money in open source you should do it by having valuable experience with some particular open source software that others are willing to pay for, not by building legal barriers to others doing the same thing. That's what I plan to do for my business and I believe Elastic can and will do it too.
My hope is that in a few years after all these "clever" attempts to build moats around open source have proven futile, people will go back to good old-fashioned experience leading the way. But maybe I shouldn't hold my breath.
You're incredibly naive and I hope you don't end up feeling screwed over by one of the big companies in a few years.
Honestly, if the open source project we maintain ever becomes popular enough that a big company like Amazon puts effort into supplanting us, great.
We'll be known as the team that created and grew the project, and we'll have a guaranteed job at Amazon and a bunch of other places doing something we have unique experience with.
My goal isn't to leverage open-source software into enough of a moat to build a billion dollar company. If it was, then I might be unhappy if Amazon broke that moat.
I'm totally happy building a sustainable small business. Maybe even one that won't last forever. Our team is small, our costs are low, and we are comfortable. We are growing, sustainably, and providing real value to our customers in the process (or they wouldn't pay us the rates we require to stay in business).
We'll be known as the team that created and grew the project, and we'll have a guaranteed job at Amazon and a bunch of other places doing something we have unique experience with.
My goal isn't to leverage open-source software into enough of a moat to build a billion dollar company. If it was, then I might be unhappy if Amazon broke that moat.
I'm totally happy building a sustainable small business. Maybe even one that won't last forever. Our team is small, our costs are low, and we are comfortable. We are growing, sustainably, and providing real value to our customers in the process (or they wouldn't pay us the rates we require to stay in business).
This is a refreshing POV, tanks for sharing. How about a model where you release with some restrictions for some time, enough to grow some customers, and then open up after a delay. Is it enough to preserve some competitive advantage while also participating in a truly open software ecosystem?
It seems like VC funding is the real key to changing the calculus. If you get as much money as Elastic did, you damn well start digging that moat and get to monetizing.
Exactly! VC funding has its place, but it also greatly restricts the types of outcomes that are favorable.
I mean, in our case, we're building a geocoder. We'd have to raise, no exaggeration, tens of billions to build a moat considering the current market leader is Google. There's no way to guarantee that outcome.
On the other hand, there is _so_ much room left after Google for other companies if they don't have to get massive. There's room for us, and at least a dozen other companies I know of that have their own take on geocoding that their own customers appreciate.
I mean, in our case, we're building a geocoder. We'd have to raise, no exaggeration, tens of billions to build a moat considering the current market leader is Google. There's no way to guarantee that outcome.
On the other hand, there is _so_ much room left after Google for other companies if they don't have to get massive. There's room for us, and at least a dozen other companies I know of that have their own take on geocoding that their own customers appreciate.
The VC funding is what appears to introduce it, but really it should be there from the start. By that I mean VC funding forces the founders to consider that simply having a viable project does not mean you have a viable product, but founders who don't realize that are on a path to this destination already before the VC turns up, the VC just accelerates it.
I recently decided to experiment with ES and prefer to evaluate things with Apache 2.0 licensing if available, even if I may pay for to license it at a later time. Elasticsearch Still maintains -oss suffixed docker images and builds that are true Apache 2.0. You can read more about this here: https://www.elastic.co/products/x-pack/open
I jotted down instructions to run the docker containers based on a few different solutions from different places: https://gist.github.com/mcescalante/6be03751c820677cf0f15c7b...
I'm sharing this because I got the vibe from the linked post in the comments and the overall marketing of "Open Distro for ES" that Elasticsearch was 'not open anymore' and had just discovered the Apache 2.0 builds within the last week.
I jotted down instructions to run the docker containers based on a few different solutions from different places: https://gist.github.com/mcescalante/6be03751c820677cf0f15c7b...
I'm sharing this because I got the vibe from the linked post in the comments and the overall marketing of "Open Distro for ES" that Elasticsearch was 'not open anymore' and had just discovered the Apache 2.0 builds within the last week.
"The maintainers of open source projects have the responsibility of keeping the source destination open to everyone and not changing the rules midstream."
Since when is anybody entitled to any free software. Projects can do what they want with their code, and while free software (the Stallman kind) is nice, if projects cite you as the main reason for their change of direction, you are intellectually dishonest if you just blame them for not being willing to do volunteer work (for you) anymore.
This, to me, looks like an abusive "look what you made me do".
Since when is anybody entitled to any free software. Projects can do what they want with their code, and while free software (the Stallman kind) is nice, if projects cite you as the main reason for their change of direction, you are intellectually dishonest if you just blame them for not being willing to do volunteer work (for you) anymore.
This, to me, looks like an abusive "look what you made me do".
No one has to "not be a dick" but the world is better when you try for the goal.
What the blog is saying is "If you build an open source tool and people put their time and effort to using it, closing more and more of the toolset to paid add-ons isn't cool".
AWS just said "Hey, we can build those ourselves and give back to the community". They volunteered and made it happen.
What the blog is saying is "If you build an open source tool and people put their time and effort to using it, closing more and more of the toolset to paid add-ons isn't cool".
AWS just said "Hey, we can build those ourselves and give back to the community". They volunteered and made it happen.
They just said "Fuck you, we have more money so we own you now".
Basically Amazon screwing Elastic. Take their code and monetize it, not giving anything back. When Elastic changed their license to prevent it, Amazon forks it.
It's the biggest risk to open source right now. They are following the strategy of "commoditize the complementary good". Since Amazon makes money selling machine cycles, they want the software to be free.
It's the biggest risk to open source right now. They are following the strategy of "commoditize the complementary good". Since Amazon makes money selling machine cycles, they want the software to be free.
Elastic was attempting to leverage open-source into proprietary vendor lock-in by blurring the line between proprietary and FOSS code. As far as I'm concerned, Amazon fighting the trojan horse is a good thing.
FOSS should not be an advertising stunt that gets walked back by coupling and interleaving FOSS with proprietary... especially not a project that already has community contributions.
There are plenty of FOSS business models. If those aren't profitable enough, go proprietary.
FOSS should not be an advertising stunt that gets walked back by coupling and interleaving FOSS with proprietary... especially not a project that already has community contributions.
There are plenty of FOSS business models. If those aren't profitable enough, go proprietary.
No, no, no. Amazon is using the software in the way it was intended, legally and ethically. It's like companies published software under a license and never gave a thought to the consequences of the license they chose. If you license permissively, be prepared for your competitors to use it!
Which is why Elastic changed their licensing, and now Amazon is playing the hapless victim.
They're not playing the hapless victim. They're doing exactly what the open source license was designed to do and support. They're forking it because they don't like the direction of the current mainters, and are releasing their own open sourced updates to it.
We've seen this happen repeatedly through many open source projects over the last couple of decades. Take the Jenkins/Hudson split, for example, or OpenWrt/LEDE, Node.js/IO.js, or to stretch even further back Mambo/Joomla.
The whole purpose of the open source license is to ensure that you are always able to patch, support, and maintain what you're running. That you're not stuck depending on a single company to keep your software running. That you're always able to add missing features as you see them, etc. (obviously, the license you choose dictates whether you have to provide those back).
By mixing up proprietary code with open source code Elastic betrayed those central ideals of open source, as have Nginx and any number of companies that have adopted this model. The reason why should be pretty obvious. By building the company model around premium modules, they have disincentivised themselves to fixing or enhancing the open source version of the software. Worse, they're not incentivised in any fashion to even accept patches that open source contributors provide that provide the missing features etc.
By way of example, there's a fix for a common issue in Nginx that has landed in their upstream commercial version of Nginx, that hasn't made it in to the open source version (at least last I saw it hadn't, and the fix had been in upstream for quite a while). Nginx ignores DNS TTLs, and can seriously trip you up unless you happen to know to use a particular combination of variables and statically configured DNS resolvers in nginx to make it DNS TTL for servers it proxies too. The company behind Nginx has had zero incentive to provide that in the open source version, or accept patches that fix that behaviour. That leaves people tripping over the same bug for years on end, until they discover, just like so many before them, those blog posts that detail the behaviour and how to save yourselves from it.
What each of these companies have done essentially makes actions like Amazon's actions here completely inevitable.
We've seen this happen repeatedly through many open source projects over the last couple of decades. Take the Jenkins/Hudson split, for example, or OpenWrt/LEDE, Node.js/IO.js, or to stretch even further back Mambo/Joomla.
The whole purpose of the open source license is to ensure that you are always able to patch, support, and maintain what you're running. That you're not stuck depending on a single company to keep your software running. That you're always able to add missing features as you see them, etc. (obviously, the license you choose dictates whether you have to provide those back).
By mixing up proprietary code with open source code Elastic betrayed those central ideals of open source, as have Nginx and any number of companies that have adopted this model. The reason why should be pretty obvious. By building the company model around premium modules, they have disincentivised themselves to fixing or enhancing the open source version of the software. Worse, they're not incentivised in any fashion to even accept patches that open source contributors provide that provide the missing features etc.
By way of example, there's a fix for a common issue in Nginx that has landed in their upstream commercial version of Nginx, that hasn't made it in to the open source version (at least last I saw it hadn't, and the fix had been in upstream for quite a while). Nginx ignores DNS TTLs, and can seriously trip you up unless you happen to know to use a particular combination of variables and statically configured DNS resolvers in nginx to make it DNS TTL for servers it proxies too. The company behind Nginx has had zero incentive to provide that in the open source version, or accept patches that fix that behaviour. That leaves people tripping over the same bug for years on end, until they discover, just like so many before them, those blog posts that detail the behaviour and how to save yourselves from it.
What each of these companies have done essentially makes actions like Amazon's actions here completely inevitable.
While I agree with you, companies should really know what a free-as-in-free-beer license means for an open source product, AWS effectively plays the helpless victim in the linked blog post. If it was a simple "we knew this was going to happen, so we forked it" they would have made a
short announcement instead of a 1500 word essay on the issue.
They are hardly playing the victim, they are moving forward in the way that is most advantageous to themselves.
Yeah but this is like eating an entire plate of free samples then asking for a refill. It's technically allowed but you're ruining for everyone else.
It's like a wizard invents an incantation for a free and infinite supply of apples. They freely give away that spell, facilitate improvements from others, encourage the world to use this great spell.
Then, someone starts selling apple sauce and it makes lots of money.
Now the wizard wants to get rich too... but their sauce isn't good enough, so they try to lock down the spell. People get upset. They helped improve the spell, why should the wizard get all the money? Or their livelihoods now depend on the spell, why are they suddenly forced to pay?
Now the apple guy decided he's going to stop playing that game, and starts supporting/improving/evangelizing a restriction-free apple incantation.
Story to be continued...
Then, someone starts selling apple sauce and it makes lots of money.
Now the wizard wants to get rich too... but their sauce isn't good enough, so they try to lock down the spell. People get upset. They helped improve the spell, why should the wizard get all the money? Or their livelihoods now depend on the spell, why are they suddenly forced to pay?
Now the apple guy decided he's going to stop playing that game, and starts supporting/improving/evangelizing a restriction-free apple incantation.
Story to be continued...
That's not exactly the allegory I'm envisioning. More like future products are going to be less open for fear of being subsumed by cloud vendors. This story of Elastic is not too dissimilar to the story posted about Docker the other day. Docker Inc is in the process of being crushed by Google via Kubernetes. It's going to potentially stifle future open source enterprise software development.
Nuh-uh. Elastic made millions selling stuff on top of open source that was licensed permissively to create a market. They knew exactly what they were getting themselves into.
> millions
A company turned a modest profit? Say it isn't so!
The guy who wrote ElasticSearch created a company to finance the development of the project. For most of their history, they released their source code under a permissive license in good faith. Amazon abused that good faith, so now Elastic continues to make the source code open and license it freely to customers and the open source community (just not competitors).
I've got no beef with Amazon, but you're kidding yourself if you compare them with Elastic in terms of ambition, greed, commitment to open source, etc.
A company turned a modest profit? Say it isn't so!
The guy who wrote ElasticSearch created a company to finance the development of the project. For most of their history, they released their source code under a permissive license in good faith. Amazon abused that good faith, so now Elastic continues to make the source code open and license it freely to customers and the open source community (just not competitors).
I've got no beef with Amazon, but you're kidding yourself if you compare them with Elastic in terms of ambition, greed, commitment to open source, etc.
Basically sounds like Redhat was the last Redhat. Docker, Mesosphere, ElasticSearch, Aquia etc be damned?
Open source and venture capital do not mix well. I’m not sure I can think of any positive examples outside of OS vendors (Redhat), but there are many negative examples come to mind, going all the way back to Eazel.
These debates go back decades.
These debates go back decades.
Open source work for hire can be a lifestyle business (it's not free until you agree to write it), but VC money wants some kind of sustained competitive advantage that drives huge scale.
I think that we are seeing open-source exemplified working in full here. And it definitely signals that being 100% for open source will come with some losses.
>Basically Amazon screwing Elastic. Take their code and monetize it, not giving anything back. When Elastic changed their license to prevent it, Amazon forks it.
I do not see how Amazon is taking advantage of Elastic here. Elastic gave an offer of open source without compensation, Amazon accepted their offer, and now Elastic isn't receiving compensation. Simple as that.
>Basically Amazon screwing Elastic. Take their code and monetize it, not giving anything back. When Elastic changed their license to prevent it, Amazon forks it.
I do not see how Amazon is taking advantage of Elastic here. Elastic gave an offer of open source without compensation, Amazon accepted their offer, and now Elastic isn't receiving compensation. Simple as that.
and no one gets to make money except Amazon.
If you don't want Amazon and the like to use it, take extra care in your licensing then. Don't just throw an MIT license on there and assume all is well.
Everyone gets to make money... the open version being maintained by Amazon you am an just copy and start your own company if you want. It’s Elastic that’s trying to limit the use of their version to prevent anyone except elastic from making money...
> It's the biggest risk to open source right now.
I don't think that's true. Perhaps it's a big risk to "open core" business models, but in my mind "open source" is not about protecting your right to monetize software. Elastic wouldn't exist without Lucene, and when Elastic was flush with over 100M in venture capital, how much do you suppose they "gave back" to the Apache Software Foundation?
There are plenty of viable FOSS business models (consulting, hosting, support, etc) that are not antithetical to my hippie pinko Stallmanesque beliefs about software, so I will not be sad to see "open core" businesses get royally forked.
I don't think that's true. Perhaps it's a big risk to "open core" business models, but in my mind "open source" is not about protecting your right to monetize software. Elastic wouldn't exist without Lucene, and when Elastic was flush with over 100M in venture capital, how much do you suppose they "gave back" to the Apache Software Foundation?
There are plenty of viable FOSS business models (consulting, hosting, support, etc) that are not antithetical to my hippie pinko Stallmanesque beliefs about software, so I will not be sad to see "open core" businesses get royally forked.
>It's the biggest risk to open source right now.
A company writing open source alternatives to proprietary licensed code is the biggest risk to open source right now?
A company writing open source alternatives to proprietary licensed code is the biggest risk to open source right now?
I read that slightly different, but I think your interpretation of it is accurate to the text.
My first read made me think it was in context to a company open sourcing code, particularly hosted software. If your product is SaaS (particularly infrastructure software), one might think twice about open sourcing it, or building it as open source.
I don't know the history of elastic though, so the above may not be an accurate interpretation given more context.
I am now curious if there is a license that allows people to profit using software, but not profit by selling software that conforms to some API. Basically gating the company to be the only company who can provide the SaaS solution.
However, I don't think that license would apply to the actual API / algorithm, just the actual source.
I think this is when it starts to bleed over into software patients.
My first read made me think it was in context to a company open sourcing code, particularly hosted software. If your product is SaaS (particularly infrastructure software), one might think twice about open sourcing it, or building it as open source.
I don't know the history of elastic though, so the above may not be an accurate interpretation given more context.
I am now curious if there is a license that allows people to profit using software, but not profit by selling software that conforms to some API. Basically gating the company to be the only company who can provide the SaaS solution.
However, I don't think that license would apply to the actual API / algorithm, just the actual source.
I think this is when it starts to bleed over into software patients.
I guess we will find out if you can defend your APIs using copyright in the Oracle vs Google case
To me, the whole point of open source is the right to fork. If you are trying to remove the right to fork, that's what seems like a threat to open source to me.
The answer is you use AGPL. The Webservice Exception to the GPL is how you get screwed by Amazon and others.
Amazon has never touched or forked an AGPL project.
Amazon has never touched or forked an AGPL project.
How AGPL helps if someone like Amazon just sell your software as service without modifications?
That won't work, since Amazon reworks and modifies applications to work as SaaS, like Kinesis:Kafka.. And then Amazon never upstreams any changes they find. So Non-A GPL licenses allow companies a free pass for SaaS.
With the AGPL, if Amazon makes a change, they MUST make source available. No ifs, ands, or buts.
With the AGPL, if Amazon makes a change, they MUST make source available. No ifs, ands, or buts.
Amazon does give back code in this case right? They are re-implementing part of Elasticsearch essentially. Unless Oracle finalizes the Win against Google on the API copyright, I think what Amazon does here is confrontal, but not very controversial.
Disclaimer: Ex-Amazon Employee.
Disclaimer: Ex-Amazon Employee.
They don't have to give anything back. That's the point. Elasticsearch is free to make proprietary code and products if they want, and they use plenty of other open-source software in their database.
90% of database vendors still don't have a cloud offering for some reason even though there's intense demand from customers which AWS is more than happy to meet.
90% of database vendors still don't have a cloud offering for some reason even though there's intense demand from customers which AWS is more than happy to meet.
Hosted ES by AWS is a good solution to regret later in life.
They have minimal tooling, poor change management, poor logging, inability to address broken state and no support what so ever for a service you paid for. I won't go back to their hosted ES.
Edit: spelling
They have minimal tooling, poor change management, poor logging, inability to address broken state and no support what so ever for a service you paid for. I won't go back to their hosted ES.
Edit: spelling
I really recommend services like Bonsai and MeasuredSearch (I know there's others, but haven't used them). They have a lot of tooling and provide a lot of great support. In fact, the support and attention to your account is probably the biggest differentiator they have to AWS tooling.
For further reading, this is also mentioned in the AWS blog: https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
Thanks for sharing; I wrote the post and also submitted the distro to HN. In most cases (apparently per HN policy), blog posts that announce new services are removed in favor of direct links to the offering.
Is it really honest to say "Keeping Open Source Open"? Elasticsearch is 100% open source, even Apache Licensed, and you are implying that this is not the case "somehow". Sure "security" is missing, but who says that this has to be part of the open source project "Elasticsearch"?
Maybe it would be more honest to say "Forcing open source backing companies to avoid open core that is hated by AWS" ;) ?
(btw: I'm not associated with Elastic except loving their open source projects)
Maybe it would be more honest to say "Forcing open source backing companies to avoid open core that is hated by AWS" ;) ?
(btw: I'm not associated with Elastic except loving their open source projects)
Ah, I probably did not correctly catch the news:
> Unfortunately, since June 2018, we have witnessed significant intermingling of proprietary code into the code base.
And you mean the x-pack stuff?
https://github.com/elastic/elasticsearch/blob/master/LICENSE...
Interesting, wasn't aware of it.
> Unfortunately, since June 2018, we have witnessed significant intermingling of proprietary code into the code base.
And you mean the x-pack stuff?
https://github.com/elastic/elasticsearch/blob/master/LICENSE...
Interesting, wasn't aware of it.
[deleted]
I think a fork/distro of ES was bound to happen.
As a heavy ES user, I've looked into getting a license to access the so-called X-Pack features, most of which are very basic and should belong in the OSS distribution anyways. Lots of those features are now available as plugins, but until now there's been no overall solution for getting the missing features.
After an almost two hour long phone call, we kind of got the pricing in a roundabout way from the salesperson; tens of thousands of dollars per year for a small cluster. We REALLY wanted to get this, which is why we dealt with the sales process, but there was just no way to justify that to our management.
As a heavy ES user, I've looked into getting a license to access the so-called X-Pack features, most of which are very basic and should belong in the OSS distribution anyways. Lots of those features are now available as plugins, but until now there's been no overall solution for getting the missing features.
After an almost two hour long phone call, we kind of got the pricing in a roundabout way from the salesperson; tens of thousands of dollars per year for a small cluster. We REALLY wanted to get this, which is why we dealt with the sales process, but there was just no way to justify that to our management.
For those curious:
- the website is copyrighted to Amazon Web Services
- all commits in the github repositories are made by AWS staff.
- the website is copyrighted to Amazon Web Services
- all commits in the github repositories are made by AWS staff.
> all commits in the github repositories are made by AWS staff.
Given that it was just made public this morning, it would be surprising if it were otherwise. The real question is what the contribution model looks like going forward. The blog post says "Contributions are welcome, as are bug reports and feature requests"[1], but of course the devil is in the details.
1 - https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
Given that it was just made public this morning, it would be surprising if it were otherwise. The real question is what the contribution model looks like going forward. The blog post says "Contributions are welcome, as are bug reports and feature requests"[1], but of course the devil is in the details.
1 - https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
If you're looking for options in the search space:
- https://github.com/toshi-search/Toshi
- https://github.com/tantivy-search/tantivy (more like lucene)
- https://github.com/blevesearch/bleve
- https://github.com/toshi-search/Toshi
- https://github.com/tantivy-search/tantivy (more like lucene)
- https://github.com/blevesearch/bleve
Title isn't quite right: "As was the case with Java and OpenJDK, our intention is not to fork Elasticsearch, and we will be making contributions back to the Apache 2.0-licensed Elasticsearch upstream project as we develop add-on enhancements to the base open source software."
Unless they assume every enhancement they put in their, uh, "not a fork" repo will make it upstream, unaltered, it's a fork.
That seems inevitable to me. There's probably something eventually that either isn't wanted by Elastic, or wanted...but implemented in some different way.
That seems inevitable to me. There's probably something eventually that either isn't wanted by Elastic, or wanted...but implemented in some different way.
It looks like Amazon and Elastic have maneuvered themselves into a lose-lose situation.
Amazon has to invest into an Open-Source fork, probably won't benefit from some of the feature development Elastic will be doing, and won't get any help running their ES-as-a-Service.
Elastic won't see any revenue of the AWS ES-as-a-Service, and has lost a lot of goodwill with (potential) customers.
I think they could have reached a compromise where AWS makes sure a share of their revenue from their ES-as-a-Service ends up at Elastic, and Elastic recommends and supports ES-as-a-Service for their customers who are on AWS.
But these are apparently not the times for compromise, so we end up in a situation that is worse for AWS, Elastic and everyone in the wider community.
Amazon has to invest into an Open-Source fork, probably won't benefit from some of the feature development Elastic will be doing, and won't get any help running their ES-as-a-Service.
Elastic won't see any revenue of the AWS ES-as-a-Service, and has lost a lot of goodwill with (potential) customers.
I think they could have reached a compromise where AWS makes sure a share of their revenue from their ES-as-a-Service ends up at Elastic, and Elastic recommends and supports ES-as-a-Service for their customers who are on AWS.
But these are apparently not the times for compromise, so we end up in a situation that is worse for AWS, Elastic and everyone in the wider community.
IMHO using "open" is just strategy to attract devs and companies that are looking for FOSS solutions, but in reality it's just another way to vendor lock-in. I hope i am wrong.
Honest question, besides not paying licenses, is there any technical reason to do this?
Honest question, besides not paying licenses, is there any technical reason to do this?
I was trying to implement ES previously but kept running into things I thought we're included "open source" but turned out to be an extra paid feature. With this I know what I'm getting.
There is no more vendor lock-in with this than with Elastic.
There is no more vendor lock-in with this than with Elastic.
Does knowing that you're running or contributing to Open Source code count? The AWS Open Source blog posted elsewhere in this topic implies that Elastic is making it hard to tell.
If you are using AWS ESS I would imagine using this distro locally and in testing will be a closer match to what's running in prod.
Exactly my point. Most likely it will be more difficult migrate to the "normal" ES.
As long as this remains cleanly open, I'm not sure why you wouldn't run this if you want to self-host. I'm not sure how this will, or won't play out for Elastic company though. I'm thinking AGPL for server, and MIT for client libraries is probably the best way to go for someone starting any kind of *Server project.
Open-Core as a business model isn't something that the likes of AWS, Azure or GCP can really tolerate if they want to integrate SaaS that the customers want. DBaaS in particular is one of the bigger advantages of cloud providers in general.
Discloser: A family member of mine does work for Elastic, but I have no knowledge of their internal practices or policies or reaction to this.
Open-Core as a business model isn't something that the likes of AWS, Azure or GCP can really tolerate if they want to integrate SaaS that the customers want. DBaaS in particular is one of the bigger advantages of cloud providers in general.
Discloser: A family member of mine does work for Elastic, but I have no knowledge of their internal practices or policies or reaction to this.
I like seeing a, hopefully, solid OSS implementation of security for ES. I understand the companies need to make money and pay developers, but I've never been very comfortable with them keeping the security features out of the OSS version.
As I long time user of ES I applaud Amazon's effort here. ES is not new and it is missing basic functionality.
I'm a long time Elasticsearch user (from before 1.0). I've also used Solr and I've built some lucene based search solutions before that as early as 2003. I'm also a customer of their hosted elastic cloud and I know several people inside that company; both former colleagues and through meetups. They do awesome stuff technically and it's a great product to use.
In short, I think Amazon doing this is a good thing. I've had the pleasure of using their hosted solution and I can't recommend it with a straight face. Key management features in the API are blocked and upgrades are a pain in the ass (as in, not supported). Support and documentation is minimal. It's not terrible but you get a much better bang for buck by using Elastic cloud. Self hosting is not something I recommend you do until you need to. It's a PITA and requires a non trivial amount of devops by someone who knows what they are doing to do it right. Rarely worth the price and hassle unless you need to run this at petabyte scale on custom hardware.
First, lets look at what this is not. This is not a fork of Elasticsearch or Kibana. If you go to the github account associated with this website, you'll see that this is simply Amazon publishing the source of plugins they use in their hosted solutions. The readme of those plugins basically instructs you to download the appropriate version of Elasticsearch from Elastic.
Second, I was there when Shay Banon announced the licensing changes at Elastic ON in early 2018. At the time I already thought this was a somewhat unfortunate move as it muddles the water in terms what is and isn't open source. Technically what they announced was shared source for things that were previously closed source. Which is better than nothing but not ideal. The whole thing was designed to get more people to try this out than was the case when you needed to get a license before you could play with it. It's basically a form of shareware.
We use Elastic Cloud, it seems all the cool stuff they talk about is mostly off limits to their basic tier users astill nd requires quite a bit of skill to get started with. I appreciate that they are trying to upsell this stuff but I seem to be doing fine with just the standard OSS features (and I know they are OSS because this is what I used before elastic cloud as well). I could switch back to self hosted and not lose much (other than convenience).
Third, I think the Elasticsearch ecosystem benefits from having outside contributors; especially from big corporate entities like Amazon. Their history is literally bootstrapping off Apache Lucene as a small group of developers and consultants. They employ core committers and one of the co-founders is a core committer as well (I worked with him before Elastic happened). They've been working together with the Solr people on improving Lucene for the past six years and have made some awesome improvements. Lucene sees regular contributions from academics as well. It's a healthy project and it is core to what Elasticsearch does. IMHO this is something they need to nurture and keep and something that is core to their culture.
The plugin ecosystem around Elasticsearch is where the action is commercially. They've built a lot of cool stuff around Elasticsearch that adds a lot of value. That too sees a lot of outside innovation. So, Amazon publishing some plugins is great. Also, given that they use it at scale, they probably have a thing or two to say about how to improve Elasticsearch and probably should be regularly creating pull requests, creating issues, etc. That's a good thing. Anything blocking that would be a bad thing
In short, Elasticsearch created a big mess by mixing open and closed source and people are working around them. Instead of fighting that, they should double down of being open source at their core and benefit from this rather than attempting to fight it. I'd recommend them to 1) re-create and market their community edition, 2) actively considering offering support for Amazon customers and other people using their technology but outside of their bubble (e.g. Graylog). Work with the community instead of against it. Everybody wins.
In short, I think Amazon doing this is a good thing. I've had the pleasure of using their hosted solution and I can't recommend it with a straight face. Key management features in the API are blocked and upgrades are a pain in the ass (as in, not supported). Support and documentation is minimal. It's not terrible but you get a much better bang for buck by using Elastic cloud. Self hosting is not something I recommend you do until you need to. It's a PITA and requires a non trivial amount of devops by someone who knows what they are doing to do it right. Rarely worth the price and hassle unless you need to run this at petabyte scale on custom hardware.
First, lets look at what this is not. This is not a fork of Elasticsearch or Kibana. If you go to the github account associated with this website, you'll see that this is simply Amazon publishing the source of plugins they use in their hosted solutions. The readme of those plugins basically instructs you to download the appropriate version of Elasticsearch from Elastic.
Second, I was there when Shay Banon announced the licensing changes at Elastic ON in early 2018. At the time I already thought this was a somewhat unfortunate move as it muddles the water in terms what is and isn't open source. Technically what they announced was shared source for things that were previously closed source. Which is better than nothing but not ideal. The whole thing was designed to get more people to try this out than was the case when you needed to get a license before you could play with it. It's basically a form of shareware.
We use Elastic Cloud, it seems all the cool stuff they talk about is mostly off limits to their basic tier users astill nd requires quite a bit of skill to get started with. I appreciate that they are trying to upsell this stuff but I seem to be doing fine with just the standard OSS features (and I know they are OSS because this is what I used before elastic cloud as well). I could switch back to self hosted and not lose much (other than convenience).
Third, I think the Elasticsearch ecosystem benefits from having outside contributors; especially from big corporate entities like Amazon. Their history is literally bootstrapping off Apache Lucene as a small group of developers and consultants. They employ core committers and one of the co-founders is a core committer as well (I worked with him before Elastic happened). They've been working together with the Solr people on improving Lucene for the past six years and have made some awesome improvements. Lucene sees regular contributions from academics as well. It's a healthy project and it is core to what Elasticsearch does. IMHO this is something they need to nurture and keep and something that is core to their culture.
The plugin ecosystem around Elasticsearch is where the action is commercially. They've built a lot of cool stuff around Elasticsearch that adds a lot of value. That too sees a lot of outside innovation. So, Amazon publishing some plugins is great. Also, given that they use it at scale, they probably have a thing or two to say about how to improve Elasticsearch and probably should be regularly creating pull requests, creating issues, etc. That's a good thing. Anything blocking that would be a bad thing
In short, Elasticsearch created a big mess by mixing open and closed source and people are working around them. Instead of fighting that, they should double down of being open source at their core and benefit from this rather than attempting to fight it. I'd recommend them to 1) re-create and market their community edition, 2) actively considering offering support for Amazon customers and other people using their technology but outside of their bubble (e.g. Graylog). Work with the community instead of against it. Everybody wins.
Is this some kind of AWS is trying to cleanup their reputation in ES? Their offer of ES is super weird. THe data node is only support 2 AZs, instead of 3. They also require using the IAM for authentication(you sign the request with access+secret keys) in other words, hard to simply `curl` it.
So if they added this, maybe they will eventually bring these into AWS ES offer, so that they can do the thing that X-Pack offer - because we cannot install plugins on AWS ES yet.
So if they added this, maybe they will eventually bring these into AWS ES offer, so that they can do the thing that X-Pack offer - because we cannot install plugins on AWS ES yet.
Of all the (many and varied) criticisms that can be levelled at the AWS Elastic Search service, integration with IAM is a strange one: it’s ‘harder to curl’, because it’s more secure.
If someone is looking for a third opensource alternative to Elastic Search, I highly recommend http://vespa.ai
We migrated from ES, and it is truly next-gen compared to ES
I'd love to hear more. Could you describe your experience and elaborate on how and why it's next-gen compared to ES?
[deleted]
WDYT:
Would it have been better for AWS to initiate Open Distro for ES as part of e.g. Apache Software Foundation, whose mantra is "community over code"?
(or if not ASF for some reason, then Eclipse or CNCF for example)
(or if not ASF for some reason, then Eclipse or CNCF for example)
Yes, and I think they should have done so, just look at the responses in this thread. It's just been launched and everybody is afraid of Amazon vendor lock-in. For many techies, just the Amazon name is scary even though this is Apache 2.0 licensed.
I couldn't be happier about this. We build on Elastic today - and there are several features that we've written into our codebase from scratch, that are available in commercial Elastic only. Quite frankly if there's a community that once again "open", and that we can even contribute some of our technology to with everyone benefiting - I'll be pretty thrilled.
In tomorrow's test I'm going to throw ~3 billion messages at Elastic 6.x and then immediately stand this up over the same database... Time to see what happens :)
In tomorrow's test I'm going to throw ~3 billion messages at Elastic 6.x and then immediately stand this up over the same database... Time to see what happens :)
I'm going to respond to myself - but so far so good. I just ingested ~14B messages into a local instance, on top of an existing Elastic.
Interestingly enough - and to Amazon's credit, the have SSL running by default, with a default admin password. Our (soon to be old) solution to this was to run nginx in front of Elastic - and do another bunch of things with stunnel for balancing. I'm happy to see default encryption.
Pretty happy to start!
Interestingly enough - and to Amazon's credit, the have SSL running by default, with a default admin password. Our (soon to be old) solution to this was to run nginx in front of Elastic - and do another bunch of things with stunnel for balancing. I'm happy to see default encryption.
Pretty happy to start!
This is a good project to happen, and kudos to AWS for doing this! However, like some in the thread, I suspect AWS has been building this for a while for their own product and Elastic's license change may have influenced the decision to open-source.
Getting a good out-of-the-box granular security has been a long overdue pain point with ElasticSearch, there have been good alternatives for the other problems it is replicating.
1.) ElastAlert - https://github.com/yelp/elastalert for alerting, 2.) ElasticSearch SQL - https://github.com/NLPchina/elasticsearch-sql for writing SQL queries.
Would very much like to hear from OP why these weren't contributed to instead of creating another alternative.
We've been working on a similar project in Golang and now well may be as good a day to open-source and put it out there: https://github.com/appbaseio/arc. It's an extremely light-weight API gateway for ElasticSearch that at the core comes with an out-of-the-box security system based on users and permissions (set granular ACLs, Rate Limits, TTL) inspired from a proprietary security system we built over the past year - https://appbase.io/features/security/.
Getting a good out-of-the-box granular security has been a long overdue pain point with ElasticSearch, there have been good alternatives for the other problems it is replicating.
1.) ElastAlert - https://github.com/yelp/elastalert for alerting, 2.) ElasticSearch SQL - https://github.com/NLPchina/elasticsearch-sql for writing SQL queries.
Would very much like to hear from OP why these weren't contributed to instead of creating another alternative.
We've been working on a similar project in Golang and now well may be as good a day to open-source and put it out there: https://github.com/appbaseio/arc. It's an extremely light-weight API gateway for ElasticSearch that at the core comes with an out-of-the-box security system based on users and permissions (set granular ACLs, Rate Limits, TTL) inspired from a proprietary security system we built over the past year - https://appbase.io/features/security/.
According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin." They're not replacing the SQL extension you linked -- they're integrating it.
[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
That's definitely good to read. And gathering from other comments, they have done something similar for Security too. :-)
Who is acting in the best interest of the open source community, Elastic or AWS?
Neither. Right now AWS seems to be acting in the best interest of the ElasticSearch community, but this doesn't extend to open source as a whole.
For the long-term, the open source community should look to projects where the center of gravity is outside of an open core or FANG company. ElasticSearch went open core a while ago, and Elastic's form of open core is more like MongoDB or Redis than it is like nginx or Docker. As for Amazon, I think I'd rank them below facebook, and I would rank facebook below google and Microsoft, as stewards of open source projects.
For the long-term, the open source community should look to projects where the center of gravity is outside of an open core or FANG company. ElasticSearch went open core a while ago, and Elastic's form of open core is more like MongoDB or Redis than it is like nginx or Docker. As for Amazon, I think I'd rank them below facebook, and I would rank facebook below google and Microsoft, as stewards of open source projects.
Hadn't seen this. Wow! https://www.nginx.com/blog/nginx-joins-f5/
My guess is that Elastic is acting the best interest of Elastic and AWS in the best interest of AWS :)
I used to like a lot Elastic, we have a rather big cluster work with the OSS version, and I really disliked how they started mixing the proprietary part with the oss one. I hope we will get a good answer from them. Like splitting again the proprietary plugins and/or open sourcing some of the security features :)
AWS, in the short term.
In the long term their actions make it impossible to build any sort of business based directly on FOSS. You have to do support, or hosting, or something else.
Guess what, AWS has a lot more money to do both of those things.
and Amazon will keep right on going sucking up code and giving nothing back.
We need aggressive licensing changes.
Hell, API copyright might be necessary to stop this nonsense. We can still have good licensing in that scenario.
In the long term their actions make it impossible to build any sort of business based directly on FOSS. You have to do support, or hosting, or something else.
Guess what, AWS has a lot more money to do both of those things.
and Amazon will keep right on going sucking up code and giving nothing back.
We need aggressive licensing changes.
Hell, API copyright might be necessary to stop this nonsense. We can still have good licensing in that scenario.
I dunno if I'd be long $ESTC at this point. Elastic took $162 million in venture funding before IPOing. In the last year their Operating Income was -$43 million. A lot of that is sales/marketing related, but at some point they need to make a profit. This open distro is certainly not going to help.
As someone who's worked at many big corps on Elasticsearch installs, I've been worried about them pushing away from OSS for some time. Many big corps want some of these "Enterprise" grade features Open Distro is offering, and if Elastic doesn't already have a foothold in them, I wager they'll be using this instead of paying.
As an aside, how come people still think these Open Source/Core commercial businesses are viable? We've seen time and again that once commercialization occurs, the OSS stuff goes right out the window. (RedHat, Docker, MongoDB, Sentry etc)
As someone who's worked at many big corps on Elasticsearch installs, I've been worried about them pushing away from OSS for some time. Many big corps want some of these "Enterprise" grade features Open Distro is offering, and if Elastic doesn't already have a foothold in them, I wager they'll be using this instead of paying.
As an aside, how come people still think these Open Source/Core commercial businesses are viable? We've seen time and again that once commercialization occurs, the OSS stuff goes right out the window. (RedHat, Docker, MongoDB, Sentry etc)
Just commenting on the aside, we should be careful about painting with an overly broad brush. Each of these companies and organizations is different, at least two you've mentioned are fully opensource afaik not open core (Redhat and Sentry). Redhat continues to do tons of opensource work across many ecosystems, kubernetes, linux kernel, gnome, etc. The Redhat sw model isn't open core, because what they sell isn't proprietary afaict its a support with updates model, that they also distribute under a different brand then the opensource (fedora/centos/openshift okd). Additionally we're talking about a multi-billion dollar company, seems like a success to me. For Sentry, their more focused on Saas platform, but afaics an on premise distribution model doesn't reference a commercial product thats differential to their opensource repos aka its not open core, afaics. [update] sentry on premise install docs https://docs.sentry.io/server/installation/
RHEL, as far as I know, is not available as a nice installable package without a subscription, at least to run in production. CENTOS was started as a separate entity to make bundling the RHEL source in to easily installable distributions more straightforward, as Red Hat made it difficult.
As far as Sentry, I'm referring to the fact that they have not cut a release of their on-premise in some time, and features have started to diverge, though they assert they will reach parity with on-prem "at some point" https://forum.sentry.io/t/when-will-a-new-version-be-tagged-... (Though again, like RedHat, the source is technically available...)
My main point is that OSS is at odds with making money.
As far as Sentry, I'm referring to the fact that they have not cut a release of their on-premise in some time, and features have started to diverge, though they assert they will reach parity with on-prem "at some point" https://forum.sentry.io/t/when-will-a-new-version-be-tagged-... (Though again, like RedHat, the source is technically available...)
My main point is that OSS is at odds with making money.
Easily installed isn't part of the opensource definition, any more than source available means opensource. Calling companies that are pure opensource, open-core is misleading and detracts from the conversation, imo. Centos was a binary distribution because Redhat made redistributions of binary RHEL harder to distribute(images/logos/binaries/names via copyright/trademark), but that was part of their business model. The source however was available under opensource licenses, and most of that was from separate upstream communities, aka its linux distro (though they have lots of other products). Re sentry, if I can pull the repo and get the same bits under OSI approved license, its opensource to me. Again opensource isn't about release management practices and nice installers (although for sentry its just a docker run/build away). They might be part of good community of practices for sure though.
I'd say opensource business models are different and potentially more difficult, but there are plenty of companies selling/supporting/building opensource. The delta seems to be most vc backed companies sponsoring opensource have different expectations on growth/value extraction from customers. aka open-core is not really about opensource, its about selling proprietary products, so I'd prefer not to mis-label those who aren't doing that.
I'd say opensource business models are different and potentially more difficult, but there are plenty of companies selling/supporting/building opensource. The delta seems to be most vc backed companies sponsoring opensource have different expectations on growth/value extraction from customers. aka open-core is not really about opensource, its about selling proprietary products, so I'd prefer not to mis-label those who aren't doing that.
We can argue all day about following the spirit of something vs the letter of the law, but all four of those companies have direct business incentives to make it difficult for those wanting to use their software for free to do so. No surprise then that all have taken steps to dissuade free users or make things more challenging. You're arguing about labels, which is beside the point.
Pure software open source projects should not accept VC money and expect to stay purely open source for long.
I was always surprised one of the cloud providers didn't release a cloud-native full-text search service. A lot of projects could benefit by a simple text search service. I always found Elasticsearch to be a little clunky and found myself wishing there was an alternative. While I don't like to be locked into a cloud vendor, having a simple text search service might be enough to sway me towards a particular vendor, even if it's proprietary.
Amazon's Elasticsearch service does simplify installation and management, but honestly I wish it were even simpler. I don't want to even think about CPU/memory sizing or the number of shards.
Amazon's Elasticsearch service does simplify installation and management, but honestly I wish it were even simpler. I don't want to even think about CPU/memory sizing or the number of shards.
Well Amazon offers CloudSearch which is a proprietary full-text search engine native to the AWS cloud.
https://aws.amazon.com/cloudsearch/
EDIT:
Just to expand on this, I also found that Google Cloud and Azure both have similar proprietary services.
https://cloud.google.com/products/search/
https://azure.microsoft.com/en-us/services/search/
https://aws.amazon.com/cloudsearch/
EDIT:
Just to expand on this, I also found that Google Cloud and Azure both have similar proprietary services.
https://cloud.google.com/products/search/
https://azure.microsoft.com/en-us/services/search/
Algolia comes to mind, no?
I'm sure there should be many, many more.
I'm sure there should be many, many more.
Amazon is forcing elastic to make money from their hosted offering, instead of their proprietary x-pack modules. Professional Services don’t scale as software do (compare AWS with Rackspace) - so I’m ignoring that income stream for now. Basically, they are trying to force Elastic into the AWS red ocean. Elastic, in response, would have to make a technology leap in order to best solve their customers problems. Examples include: Integrated data prediction, anomaly detections, clustering (online machine learning). Seamless multi region cluster deployment (with some consistency guarantee)...
'Distro' makes it sound like a linux-distribution with, in this case, Elasticsearch installed on it. From what I read on the page and on the aws blog this is not the case though?
I thought the same thing. After some searching I realized that "Open Distro" isn't actually a linux distro after all. This is a non-fork release of Elasticsearch with the proprietary parts stripped out.
Thanks for figuring it out. Shame on the publishers of the OP website for making it deliberately unclear.
In what why did they deliberately make the meaning unclear? The middle of the page says "An Apache 2.0-licensed distribution of Elasticsearch". It's saying in black and white that it's a distribution of Elasticsearch, not of an operating system.
In what way is being deliberatly unclear?
In what way is being deliberatly unclear?
Hi, I work at AWS on compute services, but spent a lot of my life building Linux distros.
For those of us with Linux distro building experience, this may be initially confusing. But calling a curated collection of software a "distro" has been used for decades outside of Linux distributions.
For example, there are multiple distributions of TeX, like TeX Live, MacTeX, etc. http://www.tug.org/interest.html#free
See also the Anaconda Distribution for Python and R: https://www.anaconda.com/distribution/ (not to be confused with the Anaconda that I know and love: https://en.wikipedia.org/wiki/Anaconda_(installer) )
For those of us with Linux distro building experience, this may be initially confusing. But calling a curated collection of software a "distro" has been used for decades outside of Linux distributions.
For example, there are multiple distributions of TeX, like TeX Live, MacTeX, etc. http://www.tug.org/interest.html#free
See also the Anaconda Distribution for Python and R: https://www.anaconda.com/distribution/ (not to be confused with the Anaconda that I know and love: https://en.wikipedia.org/wiki/Anaconda_(installer) )
"Distro" is short for distribution. Many things, not just operating systems, can have different distributions. Calling it a distro doesn't make it sound like a linux distribution at all. If it was a linux distribution it would probably talk about being an OS and not being a distribution of Elasticsearch.
Can you give an example of a popular software that uses term "distro"?
Its pretty common for certain projects. As an example, here's a dozen Kubernetes distributions, all with some value add, https://www.infoworld.com/article/3265059/12-kubernetes-dist...
At least in Kubernetes land, that lead to the foundation (CNCF) creating a certified Kubernetes conformance program, to avoid end user confusion and establish a baseline feature set.
Another example from years past, would have been openstack, with many vendors shipping/selling their own branded variants.
At least in Kubernetes land, that lead to the foundation (CNCF) creating a certified Kubernetes conformance program, to avoid end user confusion and establish a baseline feature set.
Another example from years past, would have been openstack, with many vendors shipping/selling their own branded variants.
The term is used for different distributions of Hadoop and Kubernetes, for example.
Thank you for your input. I tried to google that, but I don't see any heavy usage of "distro" along with those solutions.
For example site:https://hadoop.apache.org distro nothing significant. Also, googled "Kubernetes distro" same thing.
I can assure you (within the k8s dev community anyway) that "distro" is used to refer to companies productised versions.
Look at any of the long term support discussions, and distros will be mentioned.
Same goes for OpenStack - we refer to the different vendor products as distros (this is complicated by most of the OpenStack in a box products being produced by Linux distros, but we mean all product versions of OpenStack when we was OpenStack distro)
Look at any of the long term support discussions, and distros will be mentioned.
Same goes for OpenStack - we refer to the different vendor products as distros (this is complicated by most of the OpenStack in a box products being produced by Linux distros, but we mean all product versions of OpenStack when we was OpenStack distro)
"Distribution" is definitely used w.r.t. describing different Kubernetes offerings - e.g. https://www.cncf.io/announcement/2017/11/13/cloud-native-com...
Likewise for the Hadoop project: https://wiki.apache.org/hadoop/Distributions%20and%20Commerc...
I guess you could argue that "distro" has a distinct meaning to "distribution" with the former specifically pertaining to linux distributions, but I think most people probably just interpret it as being shorthand for the latter
Likewise for the Hadoop project: https://wiki.apache.org/hadoop/Distributions%20and%20Commerc...
I guess you could argue that "distro" has a distinct meaning to "distribution" with the former specifically pertaining to linux distributions, but I think most people probably just interpret it as being shorthand for the latter
just googling "hadoop distro", without limiting it to the hadoop website, finds lots of references to people and media outlets using distro as a shorthand.
From how they've named things, the URIs and such, it seems like this might be more a concept, and that Elasticsearch is just the first one out.
So what do we think will be next?
So what do we think will be next?
I think corretto (openjdk distro) would qualify as the first one https://aws.amazon.com/corretto/
[deleted]
Question, can I checkout the code from Github and build it with ./gradlew assemble as Apache License 2 or do I need to delete some modules before doing that?
My Twitter followers include a lot of Elasticsearch and Solr users. It will be interesting to see what they think of this from their POV as users...
https://twitter.com/otisg/status/1105146138921979905
The cloud that creates an Itunes like solution for OSS, creating a business model for OSS and earning the respect of developers will make bank like Apple did. Am hoping for one of the cloud players to adopt this strategy.
Disclaimer: Views expressed above are my own and do not reflect any opinions from my employer.
Disclaimer: Views expressed above are my own and do not reflect any opinions from my employer.
ITT: people shocked by the fact that open source licensing really is open source.
- It's Open Source for everyone* -
*except for Amazon.
*except for Amazon.
I notice that ODFE (as I am trying to make the acronym a "thing") is packaged for Docker and RPM. It would be nice if they provided the plugins/extensions separately, or otherwise had build-from-source instructions.
P.s: I think people are confused. This isn't a fork? Basically AWS should have came out and said. "Hey folks we created brand spanking new OSS plugins with sepcific funxtionality for Elastic" and you can use them!
Found this perspective interesting: https://thenewstack.io/what-the-fork-amazon/
How is elastic (the company) going to make a business going forward? I would expect that selling these enterprise add-ons (or similar ones) is a significant part of their revenue.
It depends on how much revenue elastic is getting from their hosted offering and support contracts.
What do they use for Alerting? IIRC Alerting with Elasticsearch 6 is available on Enterprise edition, not OSS. And what the OS under it actually?
They're using https://github.com/opendistro-for-elasticsearch/alerting
The main selling point of the project is providing open source alternatives to features that Elastic has kept proprietary.
The main selling point of the project is providing open source alternatives to features that Elastic has kept proprietary.
[deleted]
I see this as a net positive (https://xkcd.com/927/ notwithstanding): Elastic.co, imo, has iron-grip (in terms of licensing and features that it builds) on XPack and Elasticsearch. This incredible technology that rode on OSS wave is increasingly controlled by a for-profit organization that also acts like one. This is very similar to how Google dealt with numerous forks of Android by essentially moving key pieces to GooglePlayServices and other closed-source apps in addition to responding with shutting out manufacturers and SoC providers that weren't OHA signatories [0][1]. Amazon felt the burnt of that [2].
Apart from including/forking other community plugin projects (elasticsearch-sql, SearchGuardSSL) in open-distro [3], AWS seems to be investing to make this more "cloud-native", as well: This observation comes from PerformanceAnalyzer (released) and IndexStateManagement (request-for-comment).
- IndexStateManagement [4] is simply "never send a human to do a machine's job" automation of common yet frequent tasks required to keep elasticsearch humming along. If used in conjunction with AWS's/GCP's/Azure's managed-cloud offering, this feature might be an amazing sell for a lot of enterprises that want to set up things and have them running with as few dev-ops/engs as they can.
- PerformanceAnalyzer [5] could track multiple signals from servers running open-distro and may perform reactionary actions to either shed load, throttle requests, shift workload and so on, something similar to what MeltWater did [6].
[0] https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
[1] https://techcrunch.com/2009/12/22/google-open-when-convenien...
[2] https://www.theguardian.com/technology/2013/oct/16/amazon-se...
[3] https://news.ycombinator.com/item?id=19361234
[4] https://github.com/opendistro-for-elasticsearch/index-manage...
[5] https://opendistro.github.io/for-elasticsearch-docs/docs/pa/...
[6] https://news.ycombinator.com/item?id=18413862
Apart from including/forking other community plugin projects (elasticsearch-sql, SearchGuardSSL) in open-distro [3], AWS seems to be investing to make this more "cloud-native", as well: This observation comes from PerformanceAnalyzer (released) and IndexStateManagement (request-for-comment).
- IndexStateManagement [4] is simply "never send a human to do a machine's job" automation of common yet frequent tasks required to keep elasticsearch humming along. If used in conjunction with AWS's/GCP's/Azure's managed-cloud offering, this feature might be an amazing sell for a lot of enterprises that want to set up things and have them running with as few dev-ops/engs as they can.
- PerformanceAnalyzer [5] could track multiple signals from servers running open-distro and may perform reactionary actions to either shed load, throttle requests, shift workload and so on, something similar to what MeltWater did [6].
[0] https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
[1] https://techcrunch.com/2009/12/22/google-open-when-convenien...
[2] https://www.theguardian.com/technology/2013/oct/16/amazon-se...
[3] https://news.ycombinator.com/item?id=19361234
[4] https://github.com/opendistro-for-elasticsearch/index-manage...
[5] https://opendistro.github.io/for-elasticsearch-docs/docs/pa/...
[6] https://news.ycombinator.com/item?id=18413862
The IndexStateManagement plugin is basically the same thing as Index Lifecycle Management (ILM) which was introduced in beta form in Elasticsearch 6.6. ILM is under the Elastic license though, so it’s not free-as-in-freedom, but the feature is included in the free-as-in-beer basic license.
https://www.elastic.co/guide/en/elasticsearch/reference/6.6/...
https://www.elastic.co/guide/en/elasticsearch/reference/6.6/...
So you would rather it be controlled by a very anti-open source for profit corporation nstead?
I currently run stuff in elastic. I am now thinking that postgres:JSON may be better for me.
Found this perspective interesting around COSS vs FOSS: https://thenewstack.io/what-the-fork-amazon/
Hot take, the future of open source is just massive companies sponsoring projects to get free dev hours.
You can try to fight it, but if Amazon wants your project they will either take the code or build a compatible API. There is no defense, IP rights only favour proprietary software.
You can try to fight it, but if Amazon wants your project they will either take the code or build a compatible API. There is no defense, IP rights only favour proprietary software.
Well what about enterprise support? When customers buy an enterprise solution at elastic they get support right? what would entitle them to move to this offering.
There are other companies that offer enterprise support for Elasticsearch (e.g. https://sematext.com/support/elasticsearch-production-suppor...) as well as training an such. No loss for ES users there.
well it is not same. vendor support and third party services (aka support). you can influence the product through support and through being a direct customer.
You would think so, but that is not how it works with Elastic and lots of it customers. I have heard that from N of their (ex-)customers. I get why this happens for them. There is a certain vision and goal, and anything that doesn't align is a nuisance. This is part business part philosophy, culture, and brand you want to build in and around your business.
[deleted]
It is hosted on AWS as a service hence not open source.
The whole Amazon takes an open source product and monetizes it almost reminds me in a way of someone getting rich off an asset flip on the unity store. Just a big, giant, corporate version.
OK this sounds like bullshit. Can somebody explain what's going on?
Amazon doesn't like the direction that Elastic is headed, so they've forked ElasticSearch. They don't want to maintain and document it all by themselves, and they do want for their users to be able to run it themselves (especially for running it locally or on CI), so they've open sourced it.
Well to be fair they explicitly said it wasn't a fork, and that any work they do on it will be pushed upstream. The cynical view, I guess, is that the commits they push will be providing free features that compete with Elastic's paywalled features.
I can't find anywhere where they said it isn't a fork. I searched the article for the word fork to make sure I didn't miss anything. They said it wasn't an internally forked version meaning it isn't a private fork, and they said that their intention wasn't to create a fork.
I'm not completely sure if anything that's core to ElasticSearch is forked. The repos on GitHub are modules not including one that's called "elasticsearch" or "core" or "main" or something like that, but some seem pretty essential.
As for myself, I like to call an alternative distro a fork, and I disagree that their intention isn't to create a fork, based on my idea of a fork. They're free to have their opinion on what constitutes a fork, but I don't think they're an authority on it so I'm gonna stick to my current view of what a fork is...
I'm not completely sure if anything that's core to ElasticSearch is forked. The repos on GitHub are modules not including one that's called "elasticsearch" or "core" or "main" or something like that, but some seem pretty essential.
As for myself, I like to call an alternative distro a fork, and I disagree that their intention isn't to create a fork, based on my idea of a fork. They're free to have their opinion on what constitutes a fork, but I don't think they're an authority on it so I'm gonna stick to my current view of what a fork is...
The statement was made on the AWS blog, which was linked from an earlier post of this item I think. Anyway: https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...
> This is not a fork; we will continue to send our contributions and patches upstream to advance these projects.
> This is not a fork; we will continue to send our contributions and patches upstream to advance these projects.
I find it very ironic that AWS is behind this. Whenever I buy books from Amazon I am amazed at their broken search engines. Just search a topic on Amazon and then try to sort the results. LOL.
IIRC, most of that infrastructure is/was Oracle based and much of it going to another DB. I'm not sure if Elastic is even in the mix, or how well it scales to Amazon's levels.
Given the age of the offering, it might actually be the CloudSearch [0] product that is used by the Amazon.com search team.
FWIW, I've always found the Amazon store search to be pretty decent; normally I don't use a website's built in search system, opting for Google instead.
[0] https://aws.amazon.com/cloudsearch/
FWIW, I've always found the Amazon store search to be pretty decent; normally I don't use a website's built in search system, opting for Google instead.
[0] https://aws.amazon.com/cloudsearch/
In the old days, big companies spent gazillions to buy companies when they wanted to add their software to their product lineup.
Amazon just takes it regards of whether or not the original developer is happy or hostile.
This is going to redefine open source.
Amazon just takes it regards of whether or not the original developer is happy or hostile.
This is going to redefine open source.
I don't understand this rationale. So it's Open Source for everyone, but not for Amazon because is Amazon, so fuck Amazon?
You can't have it both ways. That defeats the whole purpose of FOSS.
You can't have it both ways. That defeats the whole purpose of FOSS.
Then maybe FOSS was a mistake.
FOSS is a huge mistake, and has been for years; it's the fumbling combination of starry-eyed Utopian visions and faux-community applied to code, with no feedback loop from reality.
Good software is paid for, by someone. The divorce from reality comes in when people realize that Big-Co (in this case Amazon) is actually abiding by the license - what should cause zero cognitive dissonance is instead reverberating against the ideals that FOSS never "grew up" to encompass.
The dream of the WWW is dying, computers are truly no more "free" than they've ever been - but they're so much cheaper that we don't really care, or even notice. A few will notice the disconnect, but cannot work together to solve it - they're too busy polishing whatever "freedom" Totem stands in its place, oblivious to the foundation built on sand; ironic, given sand's place in semiconductors.
Good software is paid for, by someone. The divorce from reality comes in when people realize that Big-Co (in this case Amazon) is actually abiding by the license - what should cause zero cognitive dissonance is instead reverberating against the ideals that FOSS never "grew up" to encompass.
The dream of the WWW is dying, computers are truly no more "free" than they've ever been - but they're so much cheaper that we don't really care, or even notice. A few will notice the disconnect, but cannot work together to solve it - they're too busy polishing whatever "freedom" Totem stands in its place, oblivious to the foundation built on sand; ironic, given sand's place in semiconductors.
> FOSS is a huge mistake, and has been for years;
Stupid conclusion. Without FOSS you're forced to pay $99.99 for text editor and email client, and it's just beginning.
Open collaborations works IRL, not only in your "utopical dreams".
Stupid conclusion. Without FOSS you're forced to pay $99.99 for text editor and email client, and it's just beginning.
Open collaborations works IRL, not only in your "utopical dreams".
Most used email client: Gmail. Freedom: 0, and you are the product.
Notepad, originally released in 1983, free as in beer since Windows 1.0 in 1985.
I'd rather pay for a text editor and email client than be the product.
Open collaboration is a joke - it's always a very small group of highly technical people, paid by someone, only useful if it can be funded as a loss leader for some other part of the business. The ideological foundations are untenable at best as very little of the actual requirements of computing (graphics cards, processors, ethernet PHYs, etc) are in any way "free" - this is what I meant by people stroking their Freedom Totem, and why the whole thing is built on sand, and that computers are too cheap for them to notice. People focus on whatever their pet project is, rather than band together to actually FREE (as in freedom) computing.
We all gave up well before the Freedom dream was achieved, and in its place we got free text editors, email clients, and a surveillance based web. It's depressing.
Notepad, originally released in 1983, free as in beer since Windows 1.0 in 1985.
I'd rather pay for a text editor and email client than be the product.
Open collaboration is a joke - it's always a very small group of highly technical people, paid by someone, only useful if it can be funded as a loss leader for some other part of the business. The ideological foundations are untenable at best as very little of the actual requirements of computing (graphics cards, processors, ethernet PHYs, etc) are in any way "free" - this is what I meant by people stroking their Freedom Totem, and why the whole thing is built on sand, and that computers are too cheap for them to notice. People focus on whatever their pet project is, rather than band together to actually FREE (as in freedom) computing.
We all gave up well before the Freedom dream was achieved, and in its place we got free text editors, email clients, and a surveillance based web. It's depressing.
I don't get it - How is this different than Oracle forking RHEL or any number of other times this has happened?
[deleted]
Dear Ladies and Gentlemen,
it would be of extraordinary interest to I believe many people in our industries if anybody with some production experience on the topic would like to write a short comparison "Elasticsearch vs. Solr", thank you very much for your attention!
In most cases it's just a matter of preference. Elastic has some extra tools like kibana an was more focused on having statistics and aggregations working well, while Solr webui is a lot more complete than the elastic one and the full text search and relevance bits are more explained. Internally both use lucene, so you are going to be able to do the same things with both.
I worked extensively with both and I prefer the way of administering Solr. Anyway I use Elastic when it involves logging and statistics thanks to kibana.
I worked extensively with both and I prefer the way of administering Solr. Anyway I use Elastic when it involves logging and statistics thanks to kibana.
Here is one good comparison from Sematext (Sematext provides both Solr and Elasticsearch consulting, support, and training, so we are familiar with both Solr and ES and have no horse in the race):
https://sematext.com/blog/solr-vs-elasticsearch-differences/
https://sematext.com/blog/solr-vs-elasticsearch-differences/
meanwhile there is this: http://solr-vs-elasticsearch.com/
But some real war stories would make the comparison much more enjoyable...
But some real war stories would make the comparison much more enjoyable...
Great idea—thanks!
> At AWS, we believe that maintainers of an open source project have a responsibility to ensure that the primary open source distribution remains open and free of proprietary code so that the community can build on the project freely, and the distribution does not advantage any one company over another.
Sections like this make me wonder if customers should demand that these announcements be delivered on video, rather than in print, so we can see whether the spokesperson can say it with a straight face.
Sections like this make me wonder if customers should demand that these announcements be delivered on video, rather than in print, so we can see whether the spokesperson can say it with a straight face.
https://aws.amazon.com/blogs/opensource/keeping-open-source-...
https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic...