Jepsen: Dgraph 1.0.2(jepsen.io)
jepsen.io
Jepsen: Dgraph 1.0.2
https://jepsen.io/analyses/dgraph-1-0-2
131 comments
I assume more testing will occur after the four bugs are addressed. Will that be published on the Jepsen blog?
Yeah, we're working on fixing the existing bugs and working with Kit Patella (https://github.com/mkcp) to further extend Jepsen's Dgraph tests.
The Snapshot Isolation bank test bug, in particular, is proving a lot harder to reproduce now -- it takes many hours (even up to 8h) to show a violation; which makes debugging harder obviously. The rest 2-3 are sort of related and easier to fix.
I doubt Jepsen blog post would be updated, but once all the remaining bugs are fixed, we'll write a blog post about it on https://blog.dgraph.io.
The Snapshot Isolation bank test bug, in particular, is proving a lot harder to reproduce now -- it takes many hours (even up to 8h) to show a violation; which makes debugging harder obviously. The rest 2-3 are sort of related and easier to fix.
I doubt Jepsen blog post would be updated, but once all the remaining bugs are fixed, we'll write a blog post about it on https://blog.dgraph.io.
No man, the bugs are open from long time. The team is no more, don't know why. Only the founder is doing commits from months.
I had so much hopes on dgraph, it was way better than other graph dbs but immature
Could that be the nature of Github? Doesn't it give credit to whomever merges pull requests?
GitHub differentiates author and the one who committed if they are not same.
Well, that's unfortunate. DGraph seemed like it was pretty cool. I wish them the best of luck.
Were there any badger related issues? There doesn't seem to be reading the article.
Yeah, we did talk about Badger a bit, but I think Dgraph's internal testing was likely going to stress it more than I could over the network. We have occasionally found storage-engine level bugs with Jepsen (Tendermint, for instance), but usually the low request rates mask race conditions in storage libraries. :)
None were found based on Jepsen testing done.
We had done a separate testing for Badger before Jepsen, where we had identified and fixed a few issues related to crashes. You can read that blog post here:
https://blog.dgraph.io/post/alice/
We had done a separate testing for Badger before Jepsen, where we had identified and fixed a few issues related to crashes. You can read that blog post here:
https://blog.dgraph.io/post/alice/
Are these fixes also applicable for OpenCypher engine in Dgraph ?
Dgraph currently does not support OpenCypher.
> Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph,
Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.
Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.
Thank you! This is the best acknowledgment of our work I've seen so far. A lot of blood, sweat, hairs, frustrations, and tears has gone into building Dgraph -- and this comment by you makes me feel proud of what we're building, and the community of people using it.
I'll echo that as well. You all are doing it right!
If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
> Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is.
(OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)
Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is.
(OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)
I've been using dgraph for a side project. I chose it because I have a really high performance requirement and I find the query language feels 'right'.
Glad to see the progress made, and bugs found and fixed, for these Jepsen tests.
As a side note, interesting to see a largely Go-based, real-world project plagued with race conditions. Quite a few bugs read "but it was done in a goroutine" or "but mutable capturing in a goroutine", and even a segfault.
Glad to see the progress made, and bugs found and fixed, for these Jepsen tests.
As a side note, interesting to see a largely Go-based, real-world project plagued with race conditions. Quite a few bugs read "but it was done in a goroutine" or "but mutable capturing in a goroutine", and even a segfault.
Go probably is the easiest language to get something working with any kind of performance, but it doesn't help much to get something working correctly. After dabbling with Rust I'm amazed that multithreaded programs work at all...
For each Jepsen release my favorite thing is the code for the tests themselves -- https://github.com/jepsen-io/jepsen/tree/master/dgraph/src/j... -- which seems surprisingly unreferenced in the post itself.
Links to the code are throughout the "test design" sections of each report. :)
FYI: There's a very recent podcast with Manish, the author of DGraph, https://www.dataengineeringpodcast.com/dgraph-with-manish-ja...
Was a set of testing tools used to build the analysis? It would be nice to be able to reproduce the results.
gotta say, if you do distributed systems, subject yourself to Jepsen analysis and publish it, you win a truck-load of points in my book.
Dgraph released 1.0.6 and replication stopped working, I gave up on dgraph. It's just not reliable and immature.
It's expected since there is only single developer making commits from months.
Dgraph has been bragging about being used by big companies. Jepsen test clearly shows that there were critical bugs and no sane company can use it in production as primary database, unless they don't care about their data.
Same could be said about almost every system tested by Jepsen, so your observation seems unfair.
[deleted]
[deleted]
Important: contrary to the marketing material, dgraph is not open source - it uses the Commons Clause we've been discussing over the past few days. Discussion here:
https://github.com/dgraph-io/dgraph/issues/2545
https://github.com/dgraph-io/dgraph/issues/2545
What does this have to do with the Jepsen test discussed here? I'm pretty tired of so many comments sections turning into philosophical debates of open source licensing terms the past few days.
Maybe this will naturally bubble down so that the far more interesting technical discussions can bubble to the top, but I fear this sub-thread will eventually drown out the on-topic discussion.
Maybe this will naturally bubble down so that the far more interesting technical discussions can bubble to the top, but I fear this sub-thread will eventually drown out the on-topic discussion.
I normally wouldn't comment, but in this case we're talking about a promotional article that leads readers down a conversion funnel. Because some steps at that conversion funnel are dishonest, I want to make sure that people are aware of what they're actually looking at.
This is fair, and it looks like the more interesting discussions have bubbled up above this, so my concern appears to have been misplaced. Cheers.
Well I guess I learned something new today, open source is apparently a term that was defined by somebody and has a counter intuitive meaning. To me it has always meant that the code is available for viewing - nothing more. I assumed this without knowing anything about the OSI.
Reference: https://opensource.org/osd
Reference: https://opensource.org/osd
Yup. Open-source is a whole philosophy. What you're talking about is called source-available software.
We changed the references to say liberally licensed. I'll try and write a bigger blog post about the reasoning behind Commons Clause and Dgraph and other companies' adoption of it.
I think it's disengenuous to say liberally licensed. Liberally licensed might apply to MIT or even Apache alone, but not when you include the Commons Clause. But it's a subjective measure, so you're not making factually incorrect statements even if you are being dishonest.
Your updated statement also says "Dgraph is available under an Apache v2.0 based liberal license", which I think is unfairly using the Apache 2.0 name without clarifying that your license in fact is not open source. The language is still dishonest.
Your updated statement also says "Dgraph is available under an Apache v2.0 based liberal license", which I think is unfairly using the Apache 2.0 name without clarifying that your license in fact is not open source. The language is still dishonest.
First, to be clear, we are the authors of this software, so our responsibility is to choose our license terms and clearly communicate them to those who want to use the software. We have done that and we are obviously not trying to mislead anyone. We think our licensing statement is easy to understand. We have not changed the Apache license terms. We have added an additional term (Commons Clause) and clearly stated that we are applying the two together.
If we had changed the Apache license, we would not have referred to it by that name. But given we have not changed the Apache terms, it seems to us more confusing to arbitrarily refer to those terms by a different name.
My take is that Apache + Commons Clause is more liberal than AGPL, something I can expand on in my blog post -- that's clear from our users, who actually appreciated the switch from AGPL to the current license.
If we had changed the Apache license, we would not have referred to it by that name. But given we have not changed the Apache terms, it seems to us more confusing to arbitrarily refer to those terms by a different name.
My take is that Apache + Commons Clause is more liberal than AGPL, something I can expand on in my blog post -- that's clear from our users, who actually appreciated the switch from AGPL to the current license.
> We have not changed the Apache license terms.
Yes, you have.
> We have added an additional term (Commons Clause) and clearly stated that we are applying the two together.
Adding the Commons Clause changes the terms of the Apache license quite substantially, which is obviously the reason you added it.
Yes, you have.
> We have added an additional term (Commons Clause) and clearly stated that we are applying the two together.
Adding the Commons Clause changes the terms of the Apache license quite substantially, which is obviously the reason you added it.
The website says "Dgraph is available under an Apache v2.0 based liberal license", and Sir_Cmpwn says it should say "Dgraph is available under an Apache v2.0 + Commons Clause license" to not be misleading. Are you saying you don't want to make that really simple change?
Curious, what did come from your communication with the Apache Foundation after they voiced their concerns about your usage of the label "Apache License"?
https://github.com/dgraph-io/dgraph/issues/2416#issuecomment...
This stuff really sours me on the otherwise fairly reasonable license and its users.
https://github.com/dgraph-io/dgraph/issues/2416#issuecomment...
This stuff really sours me on the otherwise fairly reasonable license and its users.
You are free to license your software as you wish, and I am not questioning your right to. There are three separate issues here.
>our responsibility is to choose our license terms and clearly communicate them to those who want to use the software
You are not doing this. Your website reads "Dgraph is available under an Apache v2.0 based liberal license", which is technically true but also very dishonest. Your software does not use the Apache 2.0 license, and users of it should not expect to enjoy the freedoms granted to them by the Apache 2.0 license.
>We have not changed the Apache license terms
This is not true. The Apache license explicitly grants freedoms which the following
>We have added an additional term
changes. Don't wrangle words into passable lies, you're not talking to idiots. Your wording is misleading and dishonest, which are two qualities you should be trying very hard not to associate with your brand. So far you're not doing well. No one is questioning your right to license the software as you wish. But you are not proud of your choice and are using slimy and dishonest language on your marketing material to hide your shame and trick potential users.
A separate issue is the subjective one, which is whether or not the commons clause is liberal. I think you are being dishonest here. There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal.
The third issue is whether or not using the Commons Clause is the right thing for Dgraph to do. In my opinion, it's a slap in the face to everyone who ever used or especially to anyone who ever contributed to Dgraph. In fact, as I was researching dgraph a bit more to see how you approached this issue, it also occurs to me that I can't find any discussion where existing Dgraph contributors were consulted on the matter of relicensing their work, which seems to me that your license change was not only distasteful and harmful to your project, but also illegal. Were all of the contributors consulted and their permission obtained to change the license? I'll certainly be consulting them if not.
>our responsibility is to choose our license terms and clearly communicate them to those who want to use the software
You are not doing this. Your website reads "Dgraph is available under an Apache v2.0 based liberal license", which is technically true but also very dishonest. Your software does not use the Apache 2.0 license, and users of it should not expect to enjoy the freedoms granted to them by the Apache 2.0 license.
>We have not changed the Apache license terms
This is not true. The Apache license explicitly grants freedoms which the following
>We have added an additional term
changes. Don't wrangle words into passable lies, you're not talking to idiots. Your wording is misleading and dishonest, which are two qualities you should be trying very hard not to associate with your brand. So far you're not doing well. No one is questioning your right to license the software as you wish. But you are not proud of your choice and are using slimy and dishonest language on your marketing material to hide your shame and trick potential users.
A separate issue is the subjective one, which is whether or not the commons clause is liberal. I think you are being dishonest here. There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal.
The third issue is whether or not using the Commons Clause is the right thing for Dgraph to do. In my opinion, it's a slap in the face to everyone who ever used or especially to anyone who ever contributed to Dgraph. In fact, as I was researching dgraph a bit more to see how you approached this issue, it also occurs to me that I can't find any discussion where existing Dgraph contributors were consulted on the matter of relicensing their work, which seems to me that your license change was not only distasteful and harmful to your project, but also illegal. Were all of the contributors consulted and their permission obtained to change the license? I'll certainly be consulting them if not.
This is so overdramatic and unfortunate - a company puts code out there, does good work, largely follows OSS, and this topic gets derailed.
They degrade on the 'de facto' definition of OS by saying that others may not sell it.
That's it.
Comments like yours are why I don't interact with open source at all.
They degrade on the 'de facto' definition of OS by saying that others may not sell it.
That's it.
Comments like yours are why I don't interact with open source at all.
These freedoms are important, and by removing them the whole license is practically undermined. They forgot about the rights of the users and contributors while they were busy thinking about the rights of the maintainers. To quote something I said in a private email:
>The Commons Clause nullifies pretty much any privledge granted by the original license, further rendering it useless. For example, the permission to make and publish changes to the work, or reuse the code elsewhere. Is the Commons Clause viral? If I take a small function from RedisLabs and incorporate it into my project, do my users have to pay RedisLabs to support my project? If RedisLabs decides not to, is my project now illegal to support at all? What if I want to fork the software, does my fork inherit the clause and do the same problems apply? This doesn't sound anything like the "commons" to me. Pretty much all of the rights afforded to users of open source are rendered null and void, and the mention of an open source license in the licensing terms of such software is laughable.
>The Commons Clause nullifies pretty much any privledge granted by the original license, further rendering it useless. For example, the permission to make and publish changes to the work, or reuse the code elsewhere. Is the Commons Clause viral? If I take a small function from RedisLabs and incorporate it into my project, do my users have to pay RedisLabs to support my project? If RedisLabs decides not to, is my project now illegal to support at all? What if I want to fork the software, does my fork inherit the clause and do the same problems apply? This doesn't sound anything like the "commons" to me. Pretty much all of the rights afforded to users of open source are rendered null and void, and the mention of an open source license in the licensing terms of such software is laughable.
I've had my share of beefs with Sir_Cmpwn but this post is nonsense. You can't "largely" practice an open source model. Either you do or you don't.
Dgraph was interesting to me until they said they wanted to take a vig for me sharing knowledge and practical advice about the software (because that's what consulting, an activity expressly precluded in the dishonestly-named Commons Clause, is). That? That can go straight to hell.
Being a taker of open source (because you certainly "interact with open source") is your right but has absolutely nothing to do with those of us actually expecting those draping themselves in the banner of open source to practice open source principles and to run an open source project as an open source project. That exists between your ears and your ears alone.
Dgraph was interesting to me until they said they wanted to take a vig for me sharing knowledge and practical advice about the software (because that's what consulting, an activity expressly precluded in the dishonestly-named Commons Clause, is). That? That can go straight to hell.
Being a taker of open source (because you certainly "interact with open source") is your right but has absolutely nothing to do with those of us actually expecting those draping themselves in the banner of open source to practice open source principles and to run an open source project as an open source project. That exists between your ears and your ears alone.
> I've had my share of beefs with Sir_Cmpwn but this post is nonsense. You can't "largely" practice an open source model. Either you do or you don't.
Well, given that there are like 10 different criteria to meet it very obviously is not black and white.
You can view dgraph's source. You can modify and contribute to it. You can fork it, change it, etc. You just can't make it your own product that you sell.
It is open source in many, many ways, and probably the general, colloquial way. Whining endlessly because they don't want other people to sell it, and to maintain some semblance of ownership, is what's ridiculous.
Well, given that there are like 10 different criteria to meet it very obviously is not black and white.
You can view dgraph's source. You can modify and contribute to it. You can fork it, change it, etc. You just can't make it your own product that you sell.
It is open source in many, many ways, and probably the general, colloquial way. Whining endlessly because they don't want other people to sell it, and to maintain some semblance of ownership, is what's ridiculous.
You can't accept money to talk to somebody about the open-source project. The Commons Clause literally says this! Does the level of bullshit that implies not resonate? Even if there weren't a nigh-universally accepted definition of open source software that precludes it--and it does--in what universe does that suggest openness?
The code is open. You can modify the code. You can fork the code. You can do everything you'd normally do with the code.
You can not sell the code, or services that are directly based on the code.
Sounds pretty open and reasonable.
But we'll just have to disagree I think.
You can not sell the code, or services that are directly based on the code.
Sounds pretty open and reasonable.
But we'll just have to disagree I think.
Suppose you need to fix a problem with the source code or want a new feature. With the clause they've got there, you can't find a developer or company and pay them to fix the software for you. So you can't practically get changes made to the product. So, it might as well be closed source.
[deleted]
> In fact, as I was researching dgraph a bit more to see how you approached this issue, it also occurs to me that I can't find any discussion where existing Dgraph contributors were consulted on the matter of relicensing their work, which seems to me that your license change was not only distasteful and harmful to your project, but also illegal. Were all of the contributors consulted and their permission obtained to change the license?
Wait, did they not have a CLA?
Wait, did they not have a CLA?
No, I was wrong, and later found out that they've had a CLA for a while (you only find out about this when you submit a pull request, it's not mentioned in the contribution guide).
I think liberally licensed makes perfect sense:
You can use it freely for open source and commercial applications with one, single, easy-to-understand catch:
You can't sell it or provide it as a paid service.
Is it open source? No.
Can I and you and almost everyone except Google Gloud, Amazon AWS and Microsoft Azure use it exactly as if it is open source? Yes.
Is it less hassle than the AGPL? IMO, clearly yes.
You can use it freely for open source and commercial applications with one, single, easy-to-understand catch:
You can't sell it or provide it as a paid service.
Is it open source? No.
Can I and you and almost everyone except Google Gloud, Amazon AWS and Microsoft Azure use it exactly as if it is open source? Yes.
Is it less hassle than the AGPL? IMO, clearly yes.
>Can I and you and almost everyone except Google Gloud, Amazon AWS and Microsoft Azure use it exactly as if it is open source?
No, definitely not. I elaborated on why "liberal" is inappropriate here in a different comment:
>There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal.
I also wrote here about how this affects people like you and I:
https://news.ycombinator.com/item?id=17830644
No, definitely not. I elaborated on why "liberal" is inappropriate here in a different comment:
>There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal.
I also wrote here about how this affects people like you and I:
https://news.ycombinator.com/item?id=17830644
> You can use it freely for open source and commercial applications with one, single, easy-to-understand catch:
> You can't sell it or provide it as a paid service.
The limitation in the license text is not that narrow; restricting the right to sell any product or service whose value derives “substantially” from the Commons Clause software. In legal context, “substantially” generally is a rough synonym of “nontrivially”.
You seem to be suggesting that he license intended to only restrict sales of software/services which have no substantial source of value besides the upstream software, but that's not how it's actually written.
> You can't sell it or provide it as a paid service.
The limitation in the license text is not that narrow; restricting the right to sell any product or service whose value derives “substantially” from the Commons Clause software. In legal context, “substantially” generally is a rough synonym of “nontrivially”.
You seem to be suggesting that he license intended to only restrict sales of software/services which have no substantial source of value besides the upstream software, but that's not how it's actually written.
> You seem to be suggesting that he license intended to only restrict sales of software/services which have no substantial source of value besides the upstream software, but that's not how it's actually written
As far as I understand the license clearly permits me to:
-Use the licensed software internally.
-Modify the licensed software.
-Share my modifications.
-Write software that connects direcly to the licensed software, with no restrrictions on my software (unlike AGPL)
-From this it follows that I can license my software that uses the licensed software any way I want.
-I'm not sure if I will be allowed to bundle the licensed software with my software,
-But it should be allowed to say that the user as to install the licensed software and run it for my software to work.
-Sharing the licensed software in a bundle (e.g. Linux distro) seems to be OK as long as I don't charge for it. This probably will need to be sorted out for Redhat etc.
Please correct me if any of the above is incorrect. I might have misunderstood it but this was my best understanding.
As far as I understand the license clearly permits me to:
-Use the licensed software internally.
-Modify the licensed software.
-Share my modifications.
-Write software that connects direcly to the licensed software, with no restrrictions on my software (unlike AGPL)
-From this it follows that I can license my software that uses the licensed software any way I want.
-I'm not sure if I will be allowed to bundle the licensed software with my software,
-But it should be allowed to say that the user as to install the licensed software and run it for my software to work.
-Sharing the licensed software in a bundle (e.g. Linux distro) seems to be OK as long as I don't charge for it. This probably will need to be sorted out for Redhat etc.
Please correct me if any of the above is incorrect. I might have misunderstood it but this was my best understanding.
It's liberally licensed in the sense that it's freeware, but you don't get the benefits you'd have from it being open source.
> but you don't get the benefits you'd have from it being open source.
To me it seems more correct to say:
but you don't get all the benefits you'd have from it being open source.
Again though:
- while I see where this is coming from
- and it won't hurt me directly as far as I understand
- I still have a sketchy feeling about what this will lead to in the future
To me it seems more correct to say:
but you don't get all the benefits you'd have from it being open source.
Again though:
- while I see where this is coming from
- and it won't hurt me directly as far as I understand
- I still have a sketchy feeling about what this will lead to in the future
> Is it open source? No.
Yes it is. You can read the source code.
It's not FOSS, or Open Source per the OSI definition.
Yes it is. You can read the source code.
It's not FOSS, or Open Source per the OSI definition.
> Yes it is. You can read the source code.
You can read the source of Oracle Java as well. Doesn't make it open source.
Open source has a defined meaning.
Common Clause even points out in their FAQ that it makes the software not open source.
You can read the source of Oracle Java as well. Doesn't make it open source.
Open source has a defined meaning.
Common Clause even points out in their FAQ that it makes the software not open source.
This makes is source available, which is very different from open source.
Commons Clause, GPL, FSF, etc. are not liberal though.
MIT, BSD, ZLIB, Apache2, etc. are liberal.
MIT, BSD, ZLIB, Apache2, etc. are liberal.
When did the definition of "open source" change? I always took "open source" to be an umbrella term that contained both Apache-type licenses and GPL/FSF-type licenses. I get that Commons Clause is more restrictive in ways than GPL, but GPL is definitely open source. So what makes Commons Clause not open source, if GPL is?
The definition of "open source" never changed. Apache and the GPL are both open source licenses. Appending the Commons Clause removes the zeroth freedom (FSF lingo) or the first clause (OSI lingo).
https://opensource.org/osd
https://www.gnu.org/philosophy/free-sw.en.html
https://opensource.org/osd
https://www.gnu.org/philosophy/free-sw.en.html
Which in today’s world where “we were planning to fund our lives offering professional services around a production deployment” is kindly railroaded by rock bottom AWS prices is a-ok...
That’s the server side version of the anti-tivo clause that made the GPLv3.
I see a v4 coming
That’s the server side version of the anti-tivo clause that made the GPLv3.
I see a v4 coming
Withholding judgement on whether or not using a non-open license is the appropriate choice for Dgraph, the problem persists that their language is deliberately misleading. Also, GPLv3 already prevents you from tacking on the Commons Clause, any software using any GPL-family license cannot include the Commons Clause. Finally, AGPL was explicitly designed years ago to address the case you raise, i.e. the server-side Tivo problem.
I think calling their language "deliberately misleading" is deliberately misleading when there is a representative from the project here telling you that they do not intend to mislead (thus even if it is misleading, it isn't deliberate). But I guess you just seem comfortable calling that person a liar. From reading this one thread, it seems like a bummer to me that you're assuming bad faith. But I had never heard of Dgraph until this thread, so maybe there is history that I'm naively ignoring.
Yes, I'm willing to be frank and state explicitly that I'm calling this person a liar.
What's your history with them that leads you to this conclusion? If you don't have any, and you're simply making assumptions based on their choice of licensing terms, what would your argument look like if you instead imagined them as people acting in good faith, trying to provide software to people on terms that allow them to both continue improving the software while also providing for their families?
I guess the reason I have trouble assuming bad faith with things like this is because it would be much easier for these people to just get high paying jobs at big software companies than to try to eke out a living making a useful source-provided (it would be so much easier to just use "open source" here, but that is verboten) product while arguing with people like you.
I guess the reason I have trouble assuming bad faith with things like this is because it would be much easier for these people to just get high paying jobs at big software companies than to try to eke out a living making a useful source-provided (it would be so much easier to just use "open source" here, but that is verboten) product while arguing with people like you.
I think I've explained myself quite well in the thread. My personal, subjective conclusion is of dishonesty. I cannot give you an objective measure, the evidence is in this thread and you can draw your own conclusions. Being a generally good person who is trying to put food on the table for their family is not mutually exclusive with lying and resorting to questionably ethical means of doing so.
I've (unfortunately) read the whole thread. You've jumped to an uncharitable conclusion. A bunch of people have pointed this out. You can disagree while assuming good faith. It's too bad you aren't doing that here. I agree with you that it isn't a very good choice for licensing terms.
>You can disagree while assuming good faith
Yes, but bad faith exists, and the assumption of good faith is not always appropriate. I don't think that Dgraph is operating in good faith, and the evidence is enough that I am unable to suspend my disbelief. If I were Dgraph and my actions were so severely wrong that people were assuming bad faith, I would want to quickly correct that. But in fact, most of the concerns have gone un-answered.
Yes, but bad faith exists, and the assumption of good faith is not always appropriate. I don't think that Dgraph is operating in good faith, and the evidence is enough that I am unable to suspend my disbelief. If I were Dgraph and my actions were so severely wrong that people were assuming bad faith, I would want to quickly correct that. But in fact, most of the concerns have gone un-answered.
> think calling their language "deliberately misleading" is deliberately misleading when there is a representative from the project here telling you that they do not intend to mislead (thus even if it is misleading, it isn't deliberate).
Well, yes, of you assume the honesty of the person being accused of being deliberately misleading, then that forces you to reject the claim that they are being deliberately misleading, but that is also a prime example of circular reasoning.
Well, yes, of you assume the honesty of the person being accused of being deliberately misleading, then that forces you to reject the claim that they are being deliberately misleading, but that is also a prime example of circular reasoning.
Didn't I speak directly to that in the part of my comment you didn't quote?
I guess I was being too cute in my comment, so I'll switch to frankness. As a total outsider to all this, your and others' assumption of bad faith and willingness to accuse people of outright lies here is very unconvincing and off-putting to me.
I think they're probably just trying to make a living off of software they put a lot of time into creating. It's reasonable to disagree about their approach, but you're uncharitably going further than that.
I guess I was being too cute in my comment, so I'll switch to frankness. As a total outsider to all this, your and others' assumption of bad faith and willingness to accuse people of outright lies here is very unconvincing and off-putting to me.
I think they're probably just trying to make a living off of software they put a lot of time into creating. It's reasonable to disagree about their approach, but you're uncharitably going further than that.
You should do some homework first. https://blog.dgraph.io/post/licensing/
They claim that Dgraph started with the idea that every startup should be able to have the same level of technology as run by big giants. They started with VC funding in 2016 and their goal was always to make money. That is clearly biggest lie. There is no harm in creating a company for profit but i would appreciate if they were honest.
The only reason Dgraph is open source is to get early adopters and free testers and contribution from community.
They claim that Dgraph started with the idea that every startup should be able to have the same level of technology as run by big giants. They started with VC funding in 2016 and their goal was always to make money. That is clearly biggest lie. There is no harm in creating a company for profit but i would appreciate if they were honest.
The only reason Dgraph is open source is to get early adopters and free testers and contribution from community.
I'm so confused, what part are you saying is a lie? The whole post is focused around "building a long lasting company" through having a sustainable business model. What is dishonest about that? Aiming to provide good technology to startups and aiming to make money are not contradictory!
> Also, GPLv3 already prevents you from tacking on the Commons Clause
It prevents relicensing under the Commons Clause, but I don't see that GPLv3 + Commons Clause would be impossible as an original license, though it has more inherent conflicts to create ambiguity as to actual legal effect than Apache 2.0 + Commons Clause.
It prevents relicensing under the Commons Clause, but I don't see that GPLv3 + Commons Clause would be impossible as an original license, though it has more inherent conflicts to create ambiguity as to actual legal effect than Apache 2.0 + Commons Clause.
Text quoted from GPLv3:
>All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.
>All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.
I remembered the term prohibiting adding restrictive terms, but forgot the “if you got an additional restrictive term, you can release it.”
That strictly doesn't prohibit GPLv3 + Commons Clause as an original license, but would seem to make it functionally equivalent to bare GPLv3.
That strictly doesn't prohibit GPLv3 + Commons Clause as an original license, but would seem to make it functionally equivalent to bare GPLv3.
However AGPL has it's own problems and a lot pf people like me walks straight away if anything is only available as AGPL.
Commons clause means the software works as liberal easy-to-use open source for 99.9% of us AFAIK.
Edit: is->us
Commons clause means the software works as liberal easy-to-use open source for 99.9% of us AFAIK.
Edit: is->us
> Commons clause means the software works as liberal easy-to-use open source
Common Clause is — by its own admission — not open source.
Common Clause is — by its own admission — not open source.
"works as ... open source" and "is ... not open source" are compatible statements. For instance, the handle of a screwdriver works as a hammer but is not a hammer.
You quoted half my sentence, making it much easier to refute my argument.
I think it is called a strawman.
Edit: as can be seen in my other posts I know it is not open source. As can be seen in their FAQ they know it as well.
Edit 2: attack me -> refute my argument.
I think it is called a strawman.
Edit: as can be seen in my other posts I know it is not open source. As can be seen in their FAQ they know it as well.
Edit 2: attack me -> refute my argument.
Not true. See the AGPL section in the FAQ:
https://commonsclause.com/
https://commonsclause.com/
I don't think this is an accurate depiction of the problems with AGPL. I think the motivations are more accurately that companies like Google refuse to use AGPL'd software and businesses based on open-source/source-open software want to do business with Google et al. Of course, you wrote this FAQ entry, so maybe I'm wrong.
From https://commonsclause.com:
> It [sic] this “Open Source”?
> No. [also sic]
> “Open source”, has a specific definition that was written years ago and is by the Open Source Initiative, which approves Open Source licenses. Applying the Commons Clause to an open source project will mean the source code is available, and meets many of the elements of the Open Source Definition, such as free access to source code, freedom to modify, and freedom to re-distribute, but not all of them. So to avoid confusion, it is best not to call Commons Clause software “open source.”
Additionally, "Apache 2.0 + Commons Clause" is not an Apache license. I have seen a number of people be confused by this. I hope Apache implements some guidance with regards to trademarks on this matter, mostly so I don't have to keep saying it.
> It [sic] this “Open Source”?
> No. [also sic]
> “Open source”, has a specific definition that was written years ago and is by the Open Source Initiative, which approves Open Source licenses. Applying the Commons Clause to an open source project will mean the source code is available, and meets many of the elements of the Open Source Definition, such as free access to source code, freedom to modify, and freedom to re-distribute, but not all of them. So to avoid confusion, it is best not to call Commons Clause software “open source.”
Additionally, "Apache 2.0 + Commons Clause" is not an Apache license. I have seen a number of people be confused by this. I hope Apache implements some guidance with regards to trademarks on this matter, mostly so I don't have to keep saying it.
It’s not really Apache though. That’s the problem I see with it. If you tell me you’re Apache licensed I shouldn’t be surprised when I open your license file.
You can sell GPL'd software (on its own or as a service), you can't sell Commons Cause software as a service.
The definition has not changed, "Open Source" arose as a reaction against GPL/FSF/etc. so that code could truly be open for any and all.
Open source is not open for all. Is open only to the developers. The most and the important distinction.
Anyway we should avoid creating this confusion with this new crazy license, commons clause is malicioua in ita attempt to use a well understood term in a way that twista its meaning.
Anyway we should avoid creating this confusion with this new crazy license, commons clause is malicioua in ita attempt to use a well understood term in a way that twista its meaning.
Doesn't open source just mean you have access to read and modify the software? It's the stronger term Libre that implies that you grant all distribution/selling rights to the public.
I don’t have any problems with the commons clause in principle but I think they (the commons coalition or whatever they are) should just create a new license. Like commons-apache2.0 or something. Saying you’re Apache licensed is kind of dishonest because it’s not really.
If they do I'd rather they not use the commons name, which (perhaps intentionally) causes confusion alongside Apache Commons:
https://commons.apache.org/
https://commons.apache.org/
I think Creative Commons licenses are much better known than Apache Commons. (This is the first time I've heard of the latter, for example.)
I'd rather they didn't use the Apache trademark, too; just write a clear license of your own rather than trying to hijack a license with a radically different purpose but existing goodwill and then reversing it's meaning with an add-on clause.
It is not a radically different purpose as far as I understand; it is the same purpose: an extremely permissive and business friendly license with one, intentionally very narrow exception.
> an extremely permissive and business friendly license
In software licensing a “permissive license” is a well-established term for a free or open source license that does not restrict downstream derivatives to use the same license (or the license family) for the downstream contributions, as opposed to a copyleft license which does impose such a requirement.
Apache 2.0 is, indeed, a permissive license, and that is a central purpose of its intent.
(Anything) + Commons Clause is not a permissive license, and that too is a central purpose of its intent.
> with one, intentionally very narrow exception.
The restriction in the Common Clause is not “very narrow” (or with very clear boundaries, which makes the area of legal risk larger.)
In software licensing a “permissive license” is a well-established term for a free or open source license that does not restrict downstream derivatives to use the same license (or the license family) for the downstream contributions, as opposed to a copyleft license which does impose such a requirement.
Apache 2.0 is, indeed, a permissive license, and that is a central purpose of its intent.
(Anything) + Commons Clause is not a permissive license, and that too is a central purpose of its intent.
> with one, intentionally very narrow exception.
The restriction in the Common Clause is not “very narrow” (or with very clear boundaries, which makes the area of legal risk larger.)
> The restriction in the Common Clause is not “very narrow” (or with very clear boundaries, which makes the area of legal risk larger.)
They clearly intend it to be very narrow it seems.
Can you point out any way it will restrict me or anyone else from using it internally?
BTW: It may seem I think this is all good.
I do not think that.
I do think the cases I've seen so far can be reasonable.
The bigger and more problematic picture I think I see is:
- if every project and its dog applies this that will hurt me if I rely on cloud hosted services
- if the big actors think this is a good idea and start applying thos or something worse to their currently open source software, restricing competition on hosting.
- hollowing out of the open source movement.
They clearly intend it to be very narrow it seems.
Can you point out any way it will restrict me or anyone else from using it internally?
BTW: It may seem I think this is all good.
I do not think that.
I do think the cases I've seen so far can be reasonable.
The bigger and more problematic picture I think I see is:
- if every project and its dog applies this that will hurt me if I rely on cloud hosted services
- if the big actors think this is a good idea and start applying thos or something worse to their currently open source software, restricing competition on hosting.
- hollowing out of the open source movement.
I've seen you post. I know you're smart. So answer me this: other than that, Mrs. Lincoln, how was the play?
Honestly, it seems like a pretty clear intention to co-opt a term ("commons") which has usually meant "let's share things!" to mean something entirely different. (And shame on them for doing that.)
No, open source is defined by the Open Source Definition, published by the OSI:
https://opensource.org/osd
https://opensource.org/osd
Is the contention by OSI that when they coined the label "open source" on 1998-02-03[1] that it was both unused previously and/or that they have sole claim to that label?
Because that's never been how I understood it. I'm pretty sure I understood it to be, at least in it's most encompassing sense, that the source was visible long before the time OSI says it created the label.
Edit: s/weakest/encompassing/
1: https://opensource.org/history
Because that's never been how I understood it. I'm pretty sure I understood it to be, at least in it's most encompassing sense, that the source was visible long before the time OSI says it created the label.
Edit: s/weakest/encompassing/
1: https://opensource.org/history
The generally accepted definition of open source is the OSI definition. OSI also audits and keeps a list of software which qualifies under that definition, which is accepted as the canonical list of open source licenses by everyone. Even if OSI is not your cup of tea (they're a very netural organization, so I don't see why anyone would protest their authority), the FSF is another venerable organization that provides a similar definition of open source and maintains a similar list of open source licenses. You would be hard pressed to find any other respected authority which defines open source in a manner consistent with the Commons Clause.
It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. If you want to use a proprietary license, own up to it, don't try to get the mindshare of open source when you aren't.
The ancient Greek empire might have included Italy in the past, but an Italian who says they're from Greece today is lying.
It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. If you want to use a proprietary license, own up to it, don't try to get the mindshare of open source when you aren't.
The ancient Greek empire might have included Italy in the past, but an Italian who says they're from Greece today is lying.
> It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source.
That may be, but the terminology chosen by OSI is easily misunderstood, exacerbating this problem. When I see "open source", I think mostly of the source and can I see it, possibly whether I am allowed to alter it for myself. Like an "open book" (hey, I can freely annotate my own copy).
Whether something is free to sell, or redistribute, or alter and redistribute always seemed like variations on what the easily inferred (even if incorrectly inferred) core meaning of open source.
Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. If that's true, and common usage is at odds with OSI intention, where does that leave us? I think I could easily argue either side in that case.
That may be, but the terminology chosen by OSI is easily misunderstood, exacerbating this problem. When I see "open source", I think mostly of the source and can I see it, possibly whether I am allowed to alter it for myself. Like an "open book" (hey, I can freely annotate my own copy).
Whether something is free to sell, or redistribute, or alter and redistribute always seemed like variations on what the easily inferred (even if incorrectly inferred) core meaning of open source.
Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. If that's true, and common usage is at odds with OSI intention, where does that leave us? I think I could easily argue either side in that case.
> Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license.
It leaves us with an education problem.
Ten years ago I don't recall ever seeing people confuse the term Open Source with source-code-available - but the Open Source community was much, much smaller then so I guess the people involved were all on the same page.
Now that Open Source has genuinely won (when's the last time you hard a company say they have a policy of NOT using open source?) it seems we have a new terminology problem that I was previously unaware of.
It leaves us with an education problem.
Ten years ago I don't recall ever seeing people confuse the term Open Source with source-code-available - but the Open Source community was much, much smaller then so I guess the people involved were all on the same page.
Now that Open Source has genuinely won (when's the last time you hard a company say they have a policy of NOT using open source?) it seems we have a new terminology problem that I was previously unaware of.
Yes, I think so. If you believe that it was used previously, show evidence?
I can't seem to find any, so apparently I'm misremembering.
So what were we referring to Linux and the different software running on it back then as? I was running Linux back in 1996, and I can't seem to remember any other way it was referred to.
So what were we referring to Linux and the different software running on it back then as? I was running Linux back in 1996, and I can't seem to remember any other way it was referred to.
> So what were we referring to Linux and the different software running on it back then as
Free Software. Sometimes with “free as in speech” to distinguish from the also common “free as in beer” no-cost but restrictively licensed (or, often at the time, with no express license) software.
Free Software. Sometimes with “free as in speech” to distinguish from the also common “free as in beer” no-cost but restrictively licensed (or, often at the time, with no express license) software.
I honestly don't know that there are any posts I get more excited about on HN than Jepsen analyses. The number of landmines distributed systems create and the ability to suss them out in such detail / depth is really incredible. Kudos.
Agree. But ... I sure hope that the new way of developing distributed systems doesn't follow this pattern:
1. write some code
2. submit to Jepsen
3. if errors, goto 1
The problem with this approach is obvious.
1. write some code
2. submit to Jepsen
3. if errors, goto 1
The problem with this approach is obvious.
I'd hope they find themselves one or more with a master in math on logic proving in ways that a computer can help, and actually prove that their code, at least if compiled to LLVM bytecode, is adhering to the high-level architectural proof. Considering that they'd only need to prove consitency, nothing more, and especially not every little piece of code they are writing, but just the core that handles transactional isolation, not even that the code actually resolves in all cases (being stranded with a stuck cluster that is made to handle a hard reboot without problems is better than a cluster that silently violates constraints), it should not be an infeasible goal.
The main issue is that distributed database consistency without trivial, performance-killing locking schemes is too complex to prove when writing or using any trivial, local methods based on e.g. SMT solvers or so.
If something like cockroachdb would be proved-consistent on that level, it would be used for applications currently employing pessimistic locking due to a lack of trust in their database, or scaling vertically without really needing to (there are cases which make horizontal scaling cost-prohibitive due to the dependency chains in the algorithms that can solve them, but they can be replaced most of the time).
The main issue is that distributed database consistency without trivial, performance-killing locking schemes is too complex to prove when writing or using any trivial, local methods based on e.g. SMT solvers or so.
If something like cockroachdb would be proved-consistent on that level, it would be used for applications currently employing pessimistic locking due to a lack of trust in their database, or scaling vertically without really needing to (there are cases which make horizontal scaling cost-prohibitive due to the dependency chains in the algorithms that can solve them, but they can be replaced most of the time).
it should not be an infeasible goal
There's been a lot of work on both of these problems, but right now, proving concurrent algorithms correct, and proving equivalency of those algorithms to executable code, are very much open research problems.
There's been a lot of work on both of these problems, but right now, proving concurrent algorithms correct, and proving equivalency of those algorithms to executable code, are very much open research problems.
With most of the DBs he tests, the authors have very good understandings of distributed systems, and make very carefully considered design choices. They still make mistakes though, because they’re human. Very thorough testing is simply a great way to make any software more robust, distributed systems or otherwise.
You forgot step zero, which is "find terrible example code on SO and copy it"... then 3 goes back to step 0.
Totally agreed. My one and only wish is that they might offer an RSS feed.
Oh, yeah, that's something I could put together. In the meantime, there's a mailing list and twitter you can subscribe to! https://jepsen.io
Me too. and I don't know a thing about databases and distributed systems. I guess I still have a soft spot for the snarky ones from a few years back.
Slightly tangential but dang, this is the best graph and accompanying explanations of consistency models I've ever seen: https://jepsen.io/consistency
Does anybody else miss the style of Aphyr's older posts, with animated Barbie dolls and such mocking the product under scrutiny? Examples:
https://aphyr.com/posts/317-jepsen-elasticsearch
https://aphyr.com/posts/284-jepsen-mongodb
I understand this analysis was paid for by Dgraph so GIFs and memes aren't appropriate... but I do miss them.
https://aphyr.com/posts/317-jepsen-elasticsearch
https://aphyr.com/posts/284-jepsen-mongodb
I understand this analysis was paid for by Dgraph so GIFs and memes aren't appropriate... but I do miss them.
Oh, I certainly took no shortage of flack for those early analyses, but I'm glad you appreciated them! Some of my clients have even expressed disappointment that my jepsen.io analyses for them weren't as snarky as the aphyr.com ones! It's a tough situation though--if I were to write snarky analyses, even for unpaid clients, it would raise the possibility that clients could pay for favorable treatment. So... I go through an extensive process, often involving weeks of review with my client and peers, to produce as accurate, professional, and neutral a report as I can.
There's another aspect to this: now that I've established a reputation and people take my work seriously, I don't have to be snarky or aggressive to get attention drawn to these issues. And the DB landscape has changed a lot in the last five years! Engineers want to provide formalized safety guarantees. Vendors are taking these failure modes seriously instead of dismissing them as irrelevant! Since I'm getting paid, I can invest more time into making these analyses real collaborations with the vendors. So I see my role now as more about helping people reach their safety goals, and a little less about shaming folks for making systems that didn't live up to their claims.
My other motivations--letting users know how to work with the databases they've chosen, and giving case studies to help other engineers test and improve their own systems, well, those are unchanged. :)
There's another aspect to this: now that I've established a reputation and people take my work seriously, I don't have to be snarky or aggressive to get attention drawn to these issues. And the DB landscape has changed a lot in the last five years! Engineers want to provide formalized safety guarantees. Vendors are taking these failure modes seriously instead of dismissing them as irrelevant! Since I'm getting paid, I can invest more time into making these analyses real collaborations with the vendors. So I see my role now as more about helping people reach their safety goals, and a little less about shaming folks for making systems that didn't live up to their claims.
My other motivations--letting users know how to work with the databases they've chosen, and giving case studies to help other engineers test and improve their own systems, well, those are unchanged. :)
Thank you for all of your work. I dove into those posts at the start of my career and they've shaped how I think about my work tremendously. (Along with Brendan Gregg's work!)
I still reread this and pass it around at least once our twice a year: https://aphyr.com/posts/313-strong-consistency-models
It's dense and takes some time to digest but I consider it required reading for anyone working on or with distributed systems.
I still reread this and pass it around at least once our twice a year: https://aphyr.com/posts/313-strong-consistency-models
It's dense and takes some time to digest but I consider it required reading for anyone working on or with distributed systems.
Aw, thank you! :)
> now that I've established a reputation and people take my work seriously, I don't have to be snarky or aggressive to get attention drawn to these issues.
like the parent and others here, I've lamented this. I appreciate your considered reply. This callout, in particular, is something I hadn't accounted for, but it makes perfect sense (the general idea that "we're getting serious and our clients are serious now too, so posts have to be serious" is just sort of obvious, and still likely true). Cheers to fighting the good fight!
like the parent and others here, I've lamented this. I appreciate your considered reply. This callout, in particular, is something I hadn't accounted for, but it makes perfect sense (the general idea that "we're getting serious and our clients are serious now too, so posts have to be serious" is just sort of obvious, and still likely true). Cheers to fighting the good fight!
These are great, thanks, I hadn't seen them before.
No
[deleted]
I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones.
I'm around if you have questions! Cheers, Manish