Does free software benefit from ML models being derived works of training data?(mjg59.dreamwidth.org)
mjg59.dreamwidth.org
Does free software benefit from ML models being derived works of training data?
https://mjg59.dreamwidth.org/57615.html
72 comments
seems to be similar to the argument "GitHub Copilot is not infringing copyright" https://news.ycombinator.com/item?id=27736650 : "GPL people should not push for a extension of the copyright because that will strengthen the power of giant corporations", which is completely bogus to me.
It would be good to explain what exactly is bogus. Otherwise a comment like this doesn't really say much more than "nuh uh".
I thought it was blatantly dishonest: "GPL people should ignore GPL violations because pursuing them will make them look like the giant corporations" is clearly not a defendable argument (although I might be building a strawman here)
I'm not saying we should ignore GPL violations. I'm saying we should consider whether the greater benefit for free software would be in them not being violations in the first place.
Pursuing violations is a very different thing from wanting an extension of copyright. It's very valid to want the former and not the latter.
> If software weren't copyrightable, the GPL would have no power. But this is the outcome Stallman wanted! The GPL doesn't exist because copyright is good, it exists because software being copyrightable is what enables the concept of proprietary software in the first place.
This isn't really true when software is distributed in binary form - or, increasingly, run on servers and not distributed at all. The outcome Stallman wanted is that end users would have the ability to fix bugs, and add new features, in software they were using. If software is uncopyrightable then that removes one barrier, but not necessarily the most severe one; if you're relying on cloud software whose actual implementation kept as a trade secret (for example) then you still can't fix the problems that are disrupting your workflow.
> If Github's interpretation of copyright law holds, we can train a model on proprietary code and extract concepts without having to worry about being tainted. The proprietary code itself won't enter the commons, but the ideas it embodies will. No more worries about whether you're literally copying the code that implements an algorithm you want to duplicate - simply start typing and let the model remove the risk for you.
I think the closest analogy is to source code leaks like the famous Windows 2000 one. Would having such leaks be non-copyrightable help free software? I doubt it - even a deliberately released "code dump" is still difficult to learn much from, and even free-software projects have to put a lot of time and effort into documenting their code structure, having a contributor onboarding process and so on. The value of the GPL isn't that it forces people to make their raw code available - "hostile" (L)GPL code, like early WebKit or xlvns, has never made for successful projects - the value is that it creates an environment where people are actually cooperating on software development in a positive way. I think that's more likely to happen in a world where you can only use Copilot-like tools if your software is free than in a world where no such rule exists.
This isn't really true when software is distributed in binary form - or, increasingly, run on servers and not distributed at all. The outcome Stallman wanted is that end users would have the ability to fix bugs, and add new features, in software they were using. If software is uncopyrightable then that removes one barrier, but not necessarily the most severe one; if you're relying on cloud software whose actual implementation kept as a trade secret (for example) then you still can't fix the problems that are disrupting your workflow.
> If Github's interpretation of copyright law holds, we can train a model on proprietary code and extract concepts without having to worry about being tainted. The proprietary code itself won't enter the commons, but the ideas it embodies will. No more worries about whether you're literally copying the code that implements an algorithm you want to duplicate - simply start typing and let the model remove the risk for you.
I think the closest analogy is to source code leaks like the famous Windows 2000 one. Would having such leaks be non-copyrightable help free software? I doubt it - even a deliberately released "code dump" is still difficult to learn much from, and even free-software projects have to put a lot of time and effort into documenting their code structure, having a contributor onboarding process and so on. The value of the GPL isn't that it forces people to make their raw code available - "hostile" (L)GPL code, like early WebKit or xlvns, has never made for successful projects - the value is that it creates an environment where people are actually cooperating on software development in a positive way. I think that's more likely to happen in a world where you can only use Copilot-like tools if your software is free than in a world where no such rule exists.
If software wasn't copyrightable, then a leak of (say) windows 8 would not "help free software" by allowing to learn or copy from that, but rather that it would instantly create a "free-windows-8" that could get maintained with security patches by the community, and companies could use windows 8 forever for free instead of buying windows 10 or 11 licenses.
Maybe. We'd have to figure out how to build it, and we'd have to rediscover the reasons for all of the seemingly crazy workarounds that were in the code as it currently is. In order to be able to add security patches etc. we'd first need to develop an understanding of the code, which while possible is not trivial. Even in the success stories of an open-source group adopting a previously proprietary codebase (e.g. OpenOffice, Mozilla), it takes a huge amount of work and years or decades to get the codebase to a level where it can attract outside contributors in the way that open-source-from-day-1 projects do. In fact I'm not sure I've ever seen such a project succeed without having a large base of paid contributors and outside funding to start with.
The problem with this article is that it assumes that Stallman's intent would lead to the end result that Stallman wants, and that what Stallman wants is good. I believe this is wrong, either in one way or the other.
As far as I understand it, Stallman wants users to have the right to hack on software that they use and to be protected from bad software because of that ability to hack, like right to repair protects consumers on the hardware side.
Well, if GitHub Copilot can wash away copyright, companies will quickly realize that opening their source just allows their competitors to wipe away whatever edge they get from their software by keeping changes secret. They will use that reason to close more of their software to preserve their trade secrets.
In other words, if copyright wouldn't apply to software anymore, then companies would protect their software by trade secrets instead, leading to less open source, which would lead to less user freedom to hack, especially since such trade secret controls might be used against users themselves.
So I would argue that if Stallman wants copyright to disappear, he wants the wrong thing because it conflicts with his goal of user freedom, and Copilot is a problem. If he doesn't, then he will absolutely care about Copilot because Copilot will still be a problem since licenses use copyright to protect users.
I can't stress that amount. Our copyright on our software protects our users, and as such, we should guard it jealously. Not for our sake, but for theirs.
As far as I understand it, Stallman wants users to have the right to hack on software that they use and to be protected from bad software because of that ability to hack, like right to repair protects consumers on the hardware side.
Well, if GitHub Copilot can wash away copyright, companies will quickly realize that opening their source just allows their competitors to wipe away whatever edge they get from their software by keeping changes secret. They will use that reason to close more of their software to preserve their trade secrets.
In other words, if copyright wouldn't apply to software anymore, then companies would protect their software by trade secrets instead, leading to less open source, which would lead to less user freedom to hack, especially since such trade secret controls might be used against users themselves.
So I would argue that if Stallman wants copyright to disappear, he wants the wrong thing because it conflicts with his goal of user freedom, and Copilot is a problem. If he doesn't, then he will absolutely care about Copilot because Copilot will still be a problem since licenses use copyright to protect users.
I can't stress that amount. Our copyright on our software protects our users, and as such, we should guard it jealously. Not for our sake, but for theirs.
The GPL needs copyright to work correctly, otherwise companies could legally take GPL'd code and resell it, sans source, and not violate the law.
(Arguing IP on the Internet? what is this, the 90's)
(Arguing IP on the Internet? what is this, the 90's)
GPL doesn't need copyright. It only needs the ability to say "by using this software you agree to ...".
And what follows doesn't need to be completely unrestricted. You might ban for example all clauses that restrict copying this piece of software in any way or using it as is.
And what follows doesn't need to be completely unrestricted. You might ban for example all clauses that restrict copying this piece of software in any way or using it as is.
the ability to de facto say "by using this software you agree to ..." is based on copyright - because anybody can refuse the license, saying "by using this you agree to X" is technically a lie (by the way, by reading this comment you agree to send me $1000 worth of BTC within 10 days), "using this" does not make you enter a binding contract, it's just that if you refuse to accept the license, you're infringing on copyright. The licenses are accepted because of threat of a copyright infringement lawsuit. If there was no copyright, people would be free to ignore all such license offers as they would not require a license in the first place.
Indeed, this is stated quite explicitly in the GPL itself:
> ...nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
> ...nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
Plus there's nothing in Copilot that would weaken the copyright of closed source software - pirated copies of Windows and first-run movies would remain exactly as illegal as before. The "weaken copyright" claim is some bad-reasoning I'm groping for a word to describe. It's as if Google stole stuff from several small companies and someone said "at least it weakens 'the corporations'".
This. Very much this.
> licenses use copyright to protect users
From too much freedom I presume? Or what did you mean exactly?
> if copyright wouldn't apply to software anymore, then companies would protect their software by trade secrets instead
As if they didn't do that already? What part of Photoshop is not secret? Or ar least wasn't at some point until people reverse engineered it? Copyright is the last resort. The first and most solid line of defense for commercial software is alwas secrecy, of source code, of file formats, of binaries in case of server side software, of anything they can keep secret.
> leading to less open source
Why would that lead to open source since companies keeping their software secret, closed source and copyright protected was exactly what lead to open source in the first place?
From too much freedom I presume? Or what did you mean exactly?
> if copyright wouldn't apply to software anymore, then companies would protect their software by trade secrets instead
As if they didn't do that already? What part of Photoshop is not secret? Or ar least wasn't at some point until people reverse engineered it? Copyright is the last resort. The first and most solid line of defense for commercial software is alwas secrecy, of source code, of file formats, of binaries in case of server side software, of anything they can keep secret.
> leading to less open source
Why would that lead to open source since companies keeping their software secret, closed source and copyright protected was exactly what lead to open source in the first place?
> From too much freedom I presume? Or what did you mean exactly?
From things like what Audacity did: adding spyware or other bloat that serves the company rather than the user.
Also from being taken advantage of, as right to repair has shown. If we have right to repair, it's harder for companies to engage in monopolistic practices. Open source does the same thing for software.
> Why would that lead to less open source since companies keeping their software secret, closed source and copyright protected was exactly what lead to open source in the first place?
Because right now, if a company uses open source code in their code, then they will sometimes contribute back to the original software. Without copyright, they won't because their change might reveal something about their file format or something like that.
It's because of copyright that companies have played somewhat nice when using FOSS. Without it, there is no incentive to.
From things like what Audacity did: adding spyware or other bloat that serves the company rather than the user.
Also from being taken advantage of, as right to repair has shown. If we have right to repair, it's harder for companies to engage in monopolistic practices. Open source does the same thing for software.
> Why would that lead to less open source since companies keeping their software secret, closed source and copyright protected was exactly what lead to open source in the first place?
Because right now, if a company uses open source code in their code, then they will sometimes contribute back to the original software. Without copyright, they won't because their change might reveal something about their file format or something like that.
It's because of copyright that companies have played somewhat nice when using FOSS. Without it, there is no incentive to.
What trade secret controls do you imagine could be used against users? Once you've lawfully obtained a piece of software, trade secret law doesn't protect against reverse engineering.
It does if you have "agreed" to a EULA that forbids reverse engineering. I don't know if you know this, but there are already EULA's that can reverse engineering.
If copyright didn't apply to software, why would you agree to the EULA?
Do we have a choice with some of the software we have to use?
What law would they use to stop you? Just modify the binary to skip the agreement. If they don't hold copyright over the code then you're not breaking the law in doing so.
I will be breaking contract law. EULA's still hold teeth without copyright because they fall under contract law, not copyright.
Edit: I would like to add something about licenses and contract law.
While FOSS licenses can also be contracts (https://writing.kemitchell.com/2019/03/09/Deprecation-Notice...), if copyright does not apply, then users (and companies) do not have to accept the license in order to use the software because they already have access to it since the source is open.
However, with closed source software, the user does not get access until after they accept the EULA. This means that full contract law applies to the user, regardless of copyright on software.
That is why closed source software wins if copyright does not apply: because it makes an end run around copyright anyway.
Edit: I would like to add something about licenses and contract law.
While FOSS licenses can also be contracts (https://writing.kemitchell.com/2019/03/09/Deprecation-Notice...), if copyright does not apply, then users (and companies) do not have to accept the license in order to use the software because they already have access to it since the source is open.
However, with closed source software, the user does not get access until after they accept the EULA. This means that full contract law applies to the user, regardless of copyright on software.
That is why closed source software wins if copyright does not apply: because it makes an end run around copyright anyway.
Why would I have to accept the contract to obtain the software? I can just copy it off another machine where someone else accepted the contract.
I would bet that the EULA covers that too by saying that if you copy it and give it to someone else, or allow someone else to copy it from you, that person is also under the EULA.
Just checked: the Windows EULA [0] does not allow you to copy the software. And since it is a contract, that clause does not need copyright to be in force. So the person copying it to you can be targeted by Microsoft, and if you copied Windows without that user's permission, you are probably going to be charged with a violation of the CFAA. [1]
These companies have been practicing law for longer than you. Don't underestimate them.
Also, in my opinion, you are not really arguing in good faith. That is probably because my arguments really tear your article apart, and you are trying to find some way to discredit what I am saying. You can try, but I guarantee you that companies have a lot of experience protecting their closed source software. If copyright does not apply anymore, I also guarantee you that they will find, or probably already have found, a way to protect their software without it.
Edit: The Windows agreement specifically says that using the software is an implicit agreement to the EULA, and Adobe's [2] says the same thing. Thus, even if you copy from someone else's computer, you are under the EULA.
[0]: https://www.microsoft.com/en-us/Useterms/OEM/Windows/10/UseT...
[1]: https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
[2]: https://www.adobe.com/products/eula/tools/captivate.html
Just checked: the Windows EULA [0] does not allow you to copy the software. And since it is a contract, that clause does not need copyright to be in force. So the person copying it to you can be targeted by Microsoft, and if you copied Windows without that user's permission, you are probably going to be charged with a violation of the CFAA. [1]
These companies have been practicing law for longer than you. Don't underestimate them.
Also, in my opinion, you are not really arguing in good faith. That is probably because my arguments really tear your article apart, and you are trying to find some way to discredit what I am saying. You can try, but I guarantee you that companies have a lot of experience protecting their closed source software. If copyright does not apply anymore, I also guarantee you that they will find, or probably already have found, a way to protect their software without it.
Edit: The Windows agreement specifically says that using the software is an implicit agreement to the EULA, and Adobe's [2] says the same thing. Thus, even if you copy from someone else's computer, you are under the EULA.
[0]: https://www.microsoft.com/en-us/Useterms/OEM/Windows/10/UseT...
[1]: https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
[2]: https://www.adobe.com/products/eula/tools/captivate.html
> So the person copying it to you can be targeted by Microsoft, and if you copied Windows without that user's permission, you are probably going to be charged with a violation of the CFAA.
If I'm given permission to use a computer, and I then use that access to duplicate binaries that are not subject to copyright law and which the level of access I have been greanted gives me access to, which provision of the CFAA do you think I'm violating?
> Edit: The Windows agreement specifically says that using the software is an implicit agreement to the EULA, and Adobe's [2] says the same thing. Thus, even if you copy from someone else's computer, you are under the EULA.
That's something you can enforce if use of the software is controlled by law, and you can refuse me permission to use it if I don't agree to the EULA. If the code isn't copyrightable then you have no ability to do that - I'm entitled to use it without agreeing to the EULA, and asserting that I'm implicitly agreeing to it carries as much weight as me asserting that you agree to pay me $1000 by reading this comment.
If I'm given permission to use a computer, and I then use that access to duplicate binaries that are not subject to copyright law and which the level of access I have been greanted gives me access to, which provision of the CFAA do you think I'm violating?
> Edit: The Windows agreement specifically says that using the software is an implicit agreement to the EULA, and Adobe's [2] says the same thing. Thus, even if you copy from someone else's computer, you are under the EULA.
That's something you can enforce if use of the software is controlled by law, and you can refuse me permission to use it if I don't agree to the EULA. If the code isn't copyrightable then you have no ability to do that - I'm entitled to use it without agreeing to the EULA, and asserting that I'm implicitly agreeing to it carries as much weight as me asserting that you agree to pay me $1000 by reading this comment.
> If I'm given permission to use a computer, and I then use that access to duplicate binaries that are not subject to copyright law and which the level of access I have been greanted gives me access to, which provision of the CFAA do you think I'm violating?
The CFAA covers use of a computer that extends beyond the authorized use.
> That's something you can enforce if use of the software is controlled by law, and you can refuse me permission to use it if I don't agree to the EULA. If the code isn't copyrightable then you have no ability to do that - I'm entitled to use it without agreeing to the EULA, and asserting that I'm implicitly agreeing to it carries as much weight as me asserting that you agree to pay me $1000 by reading this comment.
Good luck when a company comes after you for not agreeing to the EULA.
You and I might agree that you are entitled to use it despite the EULA, but that is not what we are discussing; we are discussing how companies will protect their software if copyright does not apply.
Saying that me reading your comment implies that I agree to pay you $1000 is wrong because of two major things:
1. You posted in a public forum.
2. I did not need to copy your comment to read it.
Windows has not posted their source code in a public forum, and the use of their software on a new machine requires copying it.
Actually, this talk of $1000 gave me an idea, so let's make a bet: if you can train a Copilot-like model with a decompiled copy of Windows and not be sued by Microsoft, I'll give you the $1000 you claim I owe you. But if they sue you, then obviously, they think there is something protecting their code, whether EULA or copyright.
Good luck with that.
Edit: Also, implied contracts are a thing. [1]
[1]: https://www.investopedia.com/terms/i/implied_contract.asp
The CFAA covers use of a computer that extends beyond the authorized use.
> That's something you can enforce if use of the software is controlled by law, and you can refuse me permission to use it if I don't agree to the EULA. If the code isn't copyrightable then you have no ability to do that - I'm entitled to use it without agreeing to the EULA, and asserting that I'm implicitly agreeing to it carries as much weight as me asserting that you agree to pay me $1000 by reading this comment.
Good luck when a company comes after you for not agreeing to the EULA.
You and I might agree that you are entitled to use it despite the EULA, but that is not what we are discussing; we are discussing how companies will protect their software if copyright does not apply.
Saying that me reading your comment implies that I agree to pay you $1000 is wrong because of two major things:
1. You posted in a public forum.
2. I did not need to copy your comment to read it.
Windows has not posted their source code in a public forum, and the use of their software on a new machine requires copying it.
Actually, this talk of $1000 gave me an idea, so let's make a bet: if you can train a Copilot-like model with a decompiled copy of Windows and not be sued by Microsoft, I'll give you the $1000 you claim I owe you. But if they sue you, then obviously, they think there is something protecting their code, whether EULA or copyright.
Good luck with that.
Edit: Also, implied contracts are a thing. [1]
[1]: https://www.investopedia.com/terms/i/implied_contract.asp
> The CFAA covers use of a computer that extends beyond the authorized use.
Right. I have valid credentials for a system which grant me read access to a bunch of binaries. Are you saying that I need explicit authorisation to copy those?
> 2. I did not need to copy your comment to read it.
Agreed! I'm not going to be able to cause you to implicitly accept a EULA by doing something you already have permission to do, just as Microsoft wouldn't be able to cause me to implicitly accept a EULA by copying a binary that isn't subject to copyright law.
> But if they sue you, then obviously, they think there is something protecting their code, whether EULA or copyright.
You seem to be conflating two different situations. At present, since Windows is considered to be under copyright, Microsoft are in a position to enforce a EULA that would forbid me from doing that, even if the model I generated would produce output that wasn't considered to be a derivative work of Windows. If software couldn't be copyrighted at all then their ability to enforce that would be significantly weakened.
Right. I have valid credentials for a system which grant me read access to a bunch of binaries. Are you saying that I need explicit authorisation to copy those?
> 2. I did not need to copy your comment to read it.
Agreed! I'm not going to be able to cause you to implicitly accept a EULA by doing something you already have permission to do, just as Microsoft wouldn't be able to cause me to implicitly accept a EULA by copying a binary that isn't subject to copyright law.
> But if they sue you, then obviously, they think there is something protecting their code, whether EULA or copyright.
You seem to be conflating two different situations. At present, since Windows is considered to be under copyright, Microsoft are in a position to enforce a EULA that would forbid me from doing that, even if the model I generated would produce output that wasn't considered to be a derivative work of Windows. If software couldn't be copyrighted at all then their ability to enforce that would be significantly weakened.
these ML models are basically a fuzzy autocomplete. If you use a third-party autocomplete indexed on proprietary code who's legally liable
A lot of the comments around Copilot seems to be under the impression that courts care about how you "write" your code.
I think this naive at best, why would they and why would the law?
If you have a piece of code that would be infringing if you wrote it in Emacs. How can it not be infringing if it was written by VSCode and Copilot? I just don't see how any court would hand down such a judgment.
What I do think that Github could get away with, is not being liable for copyright infringement in my code if I used Copilot and it gave me some code that infringed. But that would just move that liability to me. Good for Github, not so good for me.
If you have a piece of code that would be infringing if you wrote it in Emacs. How can it not be infringing if it was written by VSCode and Copilot? I just don't see how any court would hand down such a judgment.
What I do think that Github could get away with, is not being liable for copyright infringement in my code if I used Copilot and it gave me some code that infringed. But that would just move that liability to me. Good for Github, not so good for me.
It can't be violating if it's not a derived work, and if courts conclude that the output of an ML model isn't derivative of its training material (which is, I understand, the current situation in the US) then it doesn't matter how similar it looks to some other code.
A blanket "the ML model's output is not at derivative of the input" from a legal perspective. Seems wrong to me, for some types of model sure that might right. For example if you train a model on recognition pictures of houses, then sure even if the pictures you used where copyrighted, the output of that model, wouldn't be. But that that generalize to one that created pictures of houses, and started outputing copies of the input pictures, that I would be surprised if was OK. So I'll agree that for some models the output isn't a legally derived from the input, but all, no I don't think that is true.
If running the data through a ML model, removes the copyright. Then we could always train models with specific input to remove copyright on that data, and we follow through on that. We could easily remove copyright on anything, and that would, if the courts upheld that. Be the death kneel for copyright. Can't really see that happening. But maybe that is just my limited imagination.
If running the data through a ML model, removes the copyright. Then we could always train models with specific input to remove copyright on that data, and we follow through on that. We could easily remove copyright on anything, and that would, if the courts upheld that. Be the death kneel for copyright. Can't really see that happening. But maybe that is just my limited imagination.
The supposed blanked "training ML is fair use, and the output of ML is not a derivative work" precedent is this: https://en.wikipedia.org/wiki/Authors_Guild,_Inc._v._Google,...
Maybe I can't read, but it's just not there. This ruling giving precedent to train generative ML systems with any source material seems to be nothing more than a shared fiction entertained by the ML industry.
Maybe I can't read, but it's just not there. This ruling giving precedent to train generative ML systems with any source material seems to be nothing more than a shared fiction entertained by the ML industry.
A lot of the comments around Copilot seems to be under the impression that courts care about how you "write" your code.
This.
Ianal but I feel pretty sure that the content of the work is what is in question, not how it was created with regards to copyright. I think wikipedia more or less says this:
https://en.wikipedia.org/wiki/Copyright_infringement#Limitat...
This.
Ianal but I feel pretty sure that the content of the work is what is in question, not how it was created with regards to copyright. I think wikipedia more or less says this:
https://en.wikipedia.org/wiki/Copyright_infringement#Limitat...
I’m a broken record at this point, but the focus on GPL code seems secondary to the massive repetition of all-rights-reserved content. I recently tried entering “chapter one: the boy who lived” as a prompt into gpt-3, and it was happy to spit out chapter one of Harry Potter, verbatim.
Especially since all-rights-reserved content is apparently the majority on GitHub. About 80% of repositories were without a license in 2015 and I doubt it has changed drastically.
https://github.blog/2015-03-09-open-source-license-usage-on-...
https://github.blog/2015-03-09-open-source-license-usage-on-...
Yep.
So we either 1) declare this to not be copyright infringement, or 2) basically outright outlaw training of interesting ML models.
I personally believe that (1) is the way to go, and I find the whole outrage about Copilot to be essentially akin to collectively shooting ourselves in the foot in the long run.
I think we shouldn't forget why copyright exists in the first place. To quote the copyright clause of the US constitution (emphasis mine):
> "*To promote the Progress of Science and useful Arts*, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries."
If anything we already went too far with how draconian modern copyright enforcement is, up to the point that it often actively hampers our progress. Please don't make it even worse!
So we either 1) declare this to not be copyright infringement, or 2) basically outright outlaw training of interesting ML models.
I personally believe that (1) is the way to go, and I find the whole outrage about Copilot to be essentially akin to collectively shooting ourselves in the foot in the long run.
I think we shouldn't forget why copyright exists in the first place. To quote the copyright clause of the US constitution (emphasis mine):
> "*To promote the Progress of Science and useful Arts*, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries."
If anything we already went too far with how draconian modern copyright enforcement is, up to the point that it often actively hampers our progress. Please don't make it even worse!
There's a lot of excluded middle between your 1 and 2. Off the top of my head, some other options to consider...
- We could extend copyright statute to explicitly allow training ML models only under particular conditions (for example, some kind of "non-commercial use" clause).
- We could enact mechanical licensing for inputs to ML models, as exists for musical cover performances. Creators of the works used as inputs would get royalties.
- We could require asking for permission to use a work as input, but standardized generous terms once permission is granted.
- We could flip that and just require notice that you're about to use a work as input, but allow the creator a certain window to object.
- We could declare that inputs are a free-for-all, automatically non-infringing, but that you (the creator of the model) are liable for extra damages if the outputs are proven to be able to infringe.
Many, many different ways this could shake out. I'm not particularly optimistic that any of (what I see as) the better choices will come to pass, but they exist.
- We could extend copyright statute to explicitly allow training ML models only under particular conditions (for example, some kind of "non-commercial use" clause).
- We could enact mechanical licensing for inputs to ML models, as exists for musical cover performances. Creators of the works used as inputs would get royalties.
- We could require asking for permission to use a work as input, but standardized generous terms once permission is granted.
- We could flip that and just require notice that you're about to use a work as input, but allow the creator a certain window to object.
- We could declare that inputs are a free-for-all, automatically non-infringing, but that you (the creator of the model) are liable for extra damages if the outputs are proven to be able to infringe.
Many, many different ways this could shake out. I'm not particularly optimistic that any of (what I see as) the better choices will come to pass, but they exist.
> 2) basically outright outlaw training of interesting ML models.
Saying that it’s infringement doesn’t mean we’re outlawing interesting ML. These definitely aren’t the only two options. A few years ago, this kind of infringement wasn’t even possible, and now it’s just beginning. This stuff is in its infancy.
Reliability and controllability is an issue our field has to confront as applications move from research to application. This is just another instance of that. We’re all eager to deploy this stuff as soon as it can do something cool, but that’s not enough, as all the discussion around copilot demonstrates.
> I think we shouldn't forget why copyright exists in the first place… To promote the Progress of Science and useful Arts
Right. It’s to encourage creators to create things. I wrote about this elsewhere [0] that the issue of generative ML models is that, as far as incentives go, you can pay a creator for use of their things, or you can throw money at AWS and get use of something like that creator’s things. The fair use equation is ‘$$$ + my stuff = your stuff,’ except that that money doesn’t go to the creators, it goes to nvidia and aws.
I think we’ll need new copyright laws for this, to make a middle ground. I personally want to be able to exclude my works from being training data unless I license then openly enough. Just like with open code licenses where you opt in. That option doesn’t seem to exist right now.
Wouldn’t it be great if a model benefiting from all this public data had to be public itself, or something along those lines? Wouldn’t that promote the arts and sciences even more?
[0] https://news.ycombinator.com/item?id=27778042
Saying that it’s infringement doesn’t mean we’re outlawing interesting ML. These definitely aren’t the only two options. A few years ago, this kind of infringement wasn’t even possible, and now it’s just beginning. This stuff is in its infancy.
Reliability and controllability is an issue our field has to confront as applications move from research to application. This is just another instance of that. We’re all eager to deploy this stuff as soon as it can do something cool, but that’s not enough, as all the discussion around copilot demonstrates.
> I think we shouldn't forget why copyright exists in the first place… To promote the Progress of Science and useful Arts
Right. It’s to encourage creators to create things. I wrote about this elsewhere [0] that the issue of generative ML models is that, as far as incentives go, you can pay a creator for use of their things, or you can throw money at AWS and get use of something like that creator’s things. The fair use equation is ‘$$$ + my stuff = your stuff,’ except that that money doesn’t go to the creators, it goes to nvidia and aws.
I think we’ll need new copyright laws for this, to make a middle ground. I personally want to be able to exclude my works from being training data unless I license then openly enough. Just like with open code licenses where you opt in. That option doesn’t seem to exist right now.
Wouldn’t it be great if a model benefiting from all this public data had to be public itself, or something along those lines? Wouldn’t that promote the arts and sciences even more?
[0] https://news.ycombinator.com/item?id=27778042
The question I have is that if Microsoft really thinks #1 is true and the way, why didn't they train Copilot on Microsoft's code (in addition to other people's code on GitHub.com) ?
Copilot only supports "Python, JavaScript, TypeScript, Ruby, and Go" but obviously the majority of Microsoft's product code is in C++ and C# and far fewer people use those two languages.
How would you go about 1 given existing law, precedent, and circumstances.
Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another?
What do you do when random corp starts shutting down open source over ownership of code it swiped from project claiming infringement?
Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another?
What do you do when random corp starts shutting down open source over ownership of code it swiped from project claiming infringement?
> Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another?
You're acting like we don't already have something similar. We do. It's called fair use. (: I'd probably just expand on that. If the portion of the work generated by a model is insubstantial, and/or doesn't harm the interests of the original rights holder then it should be okay.
If your model can generate a single excerpt of Harry Potter does that constitute harm to J.K. Rowling? Can you use it to reasonably generate a substantial portion of the book with it? Is it a reasonable substitute for buying the book? No? Then it's okay.
Some countries in fact do already have such exceptions, e.g. Japan amended its copyright law in 2018 to explicitly allow for this. To quote the relevant law:
> "It is permissible to exploit work in any way and to the extent considered necessary ...where such exploitation is not for enjoying or causing another person to enjoy the ideas or emotions expressed in such work..."
So if you want to train an ML model on a bunch of commercial novels, without the permission of their owners, and then exploit that commercially - you can do it, and it's completely legal.
You're acting like we don't already have something similar. We do. It's called fair use. (: I'd probably just expand on that. If the portion of the work generated by a model is insubstantial, and/or doesn't harm the interests of the original rights holder then it should be okay.
If your model can generate a single excerpt of Harry Potter does that constitute harm to J.K. Rowling? Can you use it to reasonably generate a substantial portion of the book with it? Is it a reasonable substitute for buying the book? No? Then it's okay.
Some countries in fact do already have such exceptions, e.g. Japan amended its copyright law in 2018 to explicitly allow for this. To quote the relevant law:
> "It is permissible to exploit work in any way and to the extent considered necessary ...where such exploitation is not for enjoying or causing another person to enjoy the ideas or emotions expressed in such work..."
So if you want to train an ML model on a bunch of commercial novels, without the permission of their owners, and then exploit that commercially - you can do it, and it's completely legal.
Harm is weird. If I sell bootleg copies of harry potter, I'm not harming anyone in any sense but opportunity cost. Sales that would have happened didn't occur. The law sees that as harm.
Personally, I'd like not have my IP used in training ML models. If they want to incorporate my stuff into their training data, they can do it on my terms, which would be either paying to license my work, or something along the lines of a new kind of copyleft license where the model has to be public and copyleft too. By including my work anyways, I didn't get the terms I would have needed.
Scale makes it weird, because it's the a-drop-of-water-in-an-ocean issue, but harm is still real harm when it's split into a billion separate pieces of infinitesimal harm.
Personally, I'd like not have my IP used in training ML models. If they want to incorporate my stuff into their training data, they can do it on my terms, which would be either paying to license my work, or something along the lines of a new kind of copyleft license where the model has to be public and copyleft too. By including my work anyways, I didn't get the terms I would have needed.
Scale makes it weird, because it's the a-drop-of-water-in-an-ocean issue, but harm is still real harm when it's split into a billion separate pieces of infinitesimal harm.
What existing law, precedent, and circumstances demonstrate that the output of an ML model is a derived work of the training data?
The problem is probably the distinguishing line. When is it a ML model doing something and when is it a human with tools doing the same thing?
In other words, to what extent does the color of the bits matter? [1] And does the output of a ML model have a new "color"?
[1]: https://ansuz.sooke.bc.ca/entry/23
In other words, to what extent does the color of the bits matter? [1] And does the output of a ML model have a new "color"?
[1]: https://ansuz.sooke.bc.ca/entry/23
If I wrote a shell script to sample a subset of your code base and rename the identifiers I shouldn't think it would require new case law to figure that out.
I think this is a too black-and-white picture of the options that are there.
I am very surprised by this. Your experiment implies that GPT-3 has sufficient parameters to keep copies of training text.
I have been effectively using GPT-3 a lot recently because I added chapters with OpenAI APIs examples to my Common Lisp and Clojure books (you can get free copies at [1] by setting the price to Free).
At Capital One, I used LSTM models to generate AWS JSON logs for testing purposes. The original logs contained sensitive data that was not in the generated data. JSON structure and names of keys was learned and reproduced verbatim, but not the values. GPT-3 has massively more parameters than my models. BTW, I also generated spreadsheet data using GANs: similar idea, but different tech.
[1] https://leanpub.com/u/markwatson
I have been effectively using GPT-3 a lot recently because I added chapters with OpenAI APIs examples to my Common Lisp and Clojure books (you can get free copies at [1] by setting the price to Free).
At Capital One, I used LSTM models to generate AWS JSON logs for testing purposes. The original logs contained sensitive data that was not in the generated data. JSON structure and names of keys was learned and reproduced verbatim, but not the values. GPT-3 has massively more parameters than my models. BTW, I also generated spreadsheet data using GANs: similar idea, but different tech.
[1] https://leanpub.com/u/markwatson
Any chance you could share the GPT-3 settings you used to get that result? I've been playing around to try and get the same thing -- closest I've gotten is having it spit out other (correct) chapter names in the same format.
With the default parameters it’ll do it fairly reliably with this
CHAPTER ONE
THE BOY WHO LIVED
CHAPTER ONE
THE BOY WHO LIVED
Mr.GitHub has quite a bit of source-available code (i.e. the source is public but it's not free software). Would be very interesting if Copilot repeats something verbatim from that.
If I say "3.14" to you and then ask you to try to come up with two more numbers that's very likely to follow what would you say?
Someone should definitely tell J. K. Rowling about that. She might be in a position to do something about this, and she definitely needs something more useful to focus her unhappiness and frustration at.
> If Github's interpretation of copyright law holds, we can train a model on proprietary code and extract concepts without having to worry about being tainted.
IANAL, but it can be more fine grained than that. For example, courts may very well care if you obtained a copy of the source code legally before you trained the AI on it, or whether the source code copy comes from a hack or a leak.
There is a difference in a court declaring that a rental agreement's clause that no parties may be hosted by the renter is void, and someone breaking into someone elses home and hosting a party there.
Proprietary source code that you can obtain legally does exist, but it's a much smaller category than proprietary code in total.
In the worst case, this might render copyleft unenforceable while doing nothing to most proprietary code.
IANAL, but it can be more fine grained than that. For example, courts may very well care if you obtained a copy of the source code legally before you trained the AI on it, or whether the source code copy comes from a hack or a leak.
There is a difference in a court declaring that a rental agreement's clause that no parties may be hosted by the renter is void, and someone breaking into someone elses home and hosting a party there.
Proprietary source code that you can obtain legally does exist, but it's a much smaller category than proprietary code in total.
In the worst case, this might render copyleft unenforceable while doing nothing to most proprietary code.
You can obtain the Windows source code legally. Lots of companies offer this to their big customers.
https://www.microsoft.com/en-us/sharedsource/
https://www.microsoft.com/en-us/sharedsource/
This idea does not work for multiple reasons:
- You can keep things secret without copyright being involved at all. Something like the Coca-Cola recipe isn't protected by copyright, but by secrecy. No problems to apply this to proprietary software.
- SaaS and PaaS make distribution terms irrelevant and secrecy very easy.
Open source for the user lost a number of years ago, the reason companies embrace it is because it commoditizes components, i.e. lowers cost, because it's literally free stuff.
- You can keep things secret without copyright being involved at all. Something like the Coca-Cola recipe isn't protected by copyright, but by secrecy. No problems to apply this to proprietary software.
- SaaS and PaaS make distribution terms irrelevant and secrecy very easy.
Open source for the user lost a number of years ago, the reason companies embrace it is because it commoditizes components, i.e. lowers cost, because it's literally free stuff.
I find this line of argumentation unconvincing, given that open source provides the source for models to learn while closed source does not and therefore there will not be derivative works of closed source at least from a copilot use case point of view.
Nevertheless, there was a link a couple weeks ago for a NN that replicated GTA. I wonder if that is more along the lines of what the author wanted to argue? I.e. if I can have a NN learn and replicate windows, would MS be ok with it?
Nevertheless, there was a link a couple weeks ago for a NN that replicated GTA. I wonder if that is more along the lines of what the author wanted to argue? I.e. if I can have a NN learn and replicate windows, would MS be ok with it?
The majority of public repositories on Github aren't "open source" - there's no associated license that grants you any right to use or modify the code. Copilot is trained on them regardless.
They rely on the fact that you granted them this right by agreeing to their ToS which allows that use. I doubt any court would consider this illegal.
This is quite an ambitious claim. Certainly the GitHub TOS grants them some common-sense ability to copy the code you upload so that they can usefully host it. Can you point to the portion that allows them to use it for Copilot?
Because I'm pretty sure it doesn't. Section D4:
> This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service...
Because I'm pretty sure it doesn't. Section D4:
> This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service...
I didn't say otherwise? The comment I replied to said:
> there will not be derivative works of closed source at least from a copilot use case point of view.
which is inaccurate, since many public repositories on Github aren't open source and ended up in Copilot's training corpus anyway.
> there will not be derivative works of closed source at least from a copilot use case point of view.
which is inaccurate, since many public repositories on Github aren't open source and ended up in Copilot's training corpus anyway.
If Microsoft truly believes they are in the right, they should publicly (and legally) state that _Microsoft_ (and not he users of copilot) takes on all responsibility for potential future copyright infringement cases against the users of copilot. If they don't do that, it's hard to take any of their claims seriously.
Or train the model on their own proprietary code base too?
At a minimum they should do that, but that's honestly not good enough. They could decide not to sue a user of copilot if it produced MS code regardless of whether copilot produces infringing code in general. An indemnification for the customer in case of third-parties suing them over code produced by copilot is honestly the only way I could take Microsoft's position seriously.
It's something like:
Having been unable to defeat free/open software through the courts and copyright, lets try it this way -- let's see if we can, without too many people noticing, gain control of free/open source through its biggest repository with "Copilot." We'll let just enough people who are not us use it in ways that we see fit, but also probably later on will fight against "scraping our data" to retain control (like we tried to do with LinkedIn.)