Ownership of AI-Generated Code Hotly Disputed(spectrum.ieee.org)
spectrum.ieee.org
Ownership of AI-Generated Code Hotly Disputed
https://spectrum.ieee.org/ai-code-generation-ownership
204 comments
Why wouldn’t it be feasible? (Maybe this depends on what you mean by ‘feasible’.) There’s no technical reason you can’t back-track the weights and make a list of which tokens from which training data were sampled. The list might be long, it could be impractical, but that has little bearing on whether it’s technically possible, right?
The problem here happens when the same source is sampled for many tokens in a row because it’s the only match for the context. It could also happen that many tokens in a row each have a long list of sources, but when put together have a subset of sources that appear in every token’s list. That means that someone’s input is being repeated verbatim, even if the network wasn’t trying to reproduce a single source. We could prune the list of attribution sources at the expense of compute by running largest common subset algorithms, which might be sufficient for attribution tracing?
It feels like this whole question might hinge on Fair Use. The network is copying other people’s code one token at a time. We (society & copyright law) all tend to agree that’s fine when it’s a single token out of context, and we all tend to agree it’s not fine when the whole output program matches any single input source. The question naturally becomes, “where’s the line, how many tokens in a row from a single source should be allowed?”
The problem here happens when the same source is sampled for many tokens in a row because it’s the only match for the context. It could also happen that many tokens in a row each have a long list of sources, but when put together have a subset of sources that appear in every token’s list. That means that someone’s input is being repeated verbatim, even if the network wasn’t trying to reproduce a single source. We could prune the list of attribution sources at the expense of compute by running largest common subset algorithms, which might be sufficient for attribution tracing?
It feels like this whole question might hinge on Fair Use. The network is copying other people’s code one token at a time. We (society & copyright law) all tend to agree that’s fine when it’s a single token out of context, and we all tend to agree it’s not fine when the whole output program matches any single input source. The question naturally becomes, “where’s the line, how many tokens in a row from a single source should be allowed?”
The token “if” appears in a heck of a lot of inputs, and a heck of a lot of outputs. What you’re describing seems like basically building an inverted index of the input corpus and then doing a search for the output. If the answer comes back with a high relevance then you consider it “traced”. I wonder what the size of that inverted index would be compared to the size of the generative model.
That sounds exactly right to me, or at least this is one specific way to implement a solution to the problem. Good question on size. Speculating… I would guess the size of the index relates not to the size of the model, but to the size of the training data. If you built an extremely naive and straightforward uncompressed index like this, you could imagine building the complete list of known tokens, and for each one the complete list of pairs of source index and character offset into each source. If a pair of 64 bit ints was sufficient for this, you’d have 128 bits per token for the index, compared to whatever the average size token is (which I’d guess is something like 2-4 bytes uncompressed). If that napkin math is anything close to reality, then I might expect the index to be ~8x the size of the training data (with error bars of like 2x-4x on both sides). Maybe we just call it an order of magnitude?
I think you'd find that most code matches (some) other existing code.
Some code, like the famous fast inverse square root, is widely shared. Even trivial code is often just copy-pasted from a popular SO answer.
Other code is driven by something like convergent evolution. It ends up similar to other code because of the limits of well known algorithms, language syntax, APIs, boilerplate, common coding styles, etc.
In other words, I doubt most human programmers would pass a "copyright check" against all existing open source code.
Some code, like the famous fast inverse square root, is widely shared. Even trivial code is often just copy-pasted from a popular SO answer.
Other code is driven by something like convergent evolution. It ends up similar to other code because of the limits of well known algorithms, language syntax, APIs, boilerplate, common coding styles, etc.
In other words, I doubt most human programmers would pass a "copyright check" against all existing open source code.
If true, how do you think these things should reflect on copyright or AI policies? Does it matter whether “most” human written code meets the same standard that we have for AI, and if so, why?
It’s hard to discuss unverifiable claims of “most” code. A lot of code, maybe most, or maybe not, is proprietary and kept within corporate walls, so we have no idea how much is or is not copied. SO answers are allowed to be copied, by definition. (And at least with SO answers, the code is publicly accessible, and attribution tracking after the fact is closer to possible, right?)
You do have a good point that attribution for some code could be practically impossible to track. This might get a lot worse if we allow AI to remix and republish it, that could even cause feedback loops if we’re not more careful about tracking attributions.
It’s hard to discuss unverifiable claims of “most” code. A lot of code, maybe most, or maybe not, is proprietary and kept within corporate walls, so we have no idea how much is or is not copied. SO answers are allowed to be copied, by definition. (And at least with SO answers, the code is publicly accessible, and attribution tracking after the fact is closer to possible, right?)
You do have a good point that attribution for some code could be practically impossible to track. This might get a lot worse if we allow AI to remix and republish it, that could even cause feedback loops if we’re not more careful about tracking attributions.
Copyright requires an element of creativity and originality. The easiest code snippets for AI to reproduce are things like language syntax (loops, definitions, etc), implementations of common algorithms, and boilerplate, and an index would probably find a lot of matches for those, but are they original or creative?
So the challenge for an index would be finding a rare case of creative, important code in a sea of trivial matches.
Incidentally, answers on SO are licensed cc-by-sa, so it's easy to violate copyright when copying them; but no one seems to care, suggesting that small code snippets are indeed "trivial".
So the challenge for an index would be finding a rare case of creative, important code in a sea of trivial matches.
Incidentally, answers on SO are licensed cc-by-sa, so it's easy to violate copyright when copying them; but no one seems to care, suggesting that small code snippets are indeed "trivial".
> There’s no technical reason you can’t back-track the weights and make a list of which tokens from which training data were sampled.
The weights map inputs to outputs, not training samples to outputs. They are adjusted during the training phase and then kept fixed (until there's a new iteration of the model; and let's ignore online learning for simplicity's sake now, the major models we're talking about here don't use it afaik). From the perspective of a user who inputs prompts and receives answers, the weights are constant. We could create some score indicating how much each training sample contributed to the weights overall, but then that score would be the same for all prompts.
It's really not clear to me what you mean by 'sampling from training data' for a specific prompt. The best I could imagine would be creating another metric measuring the distance between the prompt and all training samples, and then somehow combining that with the first score to get an overall contribution metric for this prompt. But that would be a) quite a bit of guesswork with a lot of modelling freedom and b) quite different from what you described.
The weights map inputs to outputs, not training samples to outputs. They are adjusted during the training phase and then kept fixed (until there's a new iteration of the model; and let's ignore online learning for simplicity's sake now, the major models we're talking about here don't use it afaik). From the perspective of a user who inputs prompts and receives answers, the weights are constant. We could create some score indicating how much each training sample contributed to the weights overall, but then that score would be the same for all prompts.
It's really not clear to me what you mean by 'sampling from training data' for a specific prompt. The best I could imagine would be creating another metric measuring the distance between the prompt and all training samples, and then somehow combining that with the first score to get an overall contribution metric for this prompt. But that would be a) quite a bit of guesswork with a lot of modelling freedom and b) quite different from what you described.
Fair use is a bit more context dependent, especially when applied on other form of media. Unstable diffusion is a big example where many seems to feel that no fair use should be allowed regardless of how small of a token is taken by the AI. A small amount also depend on the original work and how it fit into society at large.
Having fair use to be the pillar that all AI training stand on is going to take a while.
Having fair use to be the pillar that all AI training stand on is going to take a while.
Yeah good point, TBH I feel like the most practical way forward right now isn’t to wait for Fair Use to clarify AI, it’s probably going to take training data that’s better curated and partitioned into groups that explicitly allow certain kinds of ML use, and models that aren’t leaky across partitions (which sounds like what the problem is with Copilot in this case). This might mean new kinds of code licenses that specify whether it can be used for ML training or not, and companies are going to need to start respecting those licenses rather than assuming they can train on everything reachable from the internet.
Otherwise, if we are going to let companies ignore copyright and licenses because of their pinky promise that their models won’t be recognizable copies of any single input, then we really do need a legal criteria for how much of a single input is fair game, right?
Otherwise, if we are going to let companies ignore copyright and licenses because of their pinky promise that their models won’t be recognizable copies of any single input, then we really do need a legal criteria for how much of a single input is fair game, right?
The judge/law will say it comes down to intent.
Here's my take from an audio DSP viewpoint.
A composer listens to 10,000 hours of music. One day she writes an original piece based on the annealed parameters (temporal, spectral, intensity, pitch, sequence...) of a million artists. However it sounds like another specific artist who sues.
It is not a cover, a remix, homage, or even forged in the genre... it's just accidentally a bit too like it. (compare: Banana Splits vs. Bob Marley - and - Huey Lewis vs. Ray Parker Jr. - which completely misses the real impact of "Pop Muzik" by M)
The question is, was she exposed to influencing materials incorporated without intent or was it plagiarism (intent to reproduce a derivative etc) ?
By contrast I can take a piece of music, break it down by analysis into melodies, chords, timings, and use FFT to extract the precise spectrums of instruments, feed those to a resynthesis engine that finds new synthesiser parameters to create an exact sound-alike and then deliberately recreate a piece "In the near style of artist X". With a little musical processing I can change the key, inversions, re-template the rhythm to a new swing... always pushing the derived piece into new territory until eventually it's barely recognisable.
Nonetheless, in the second case I have clearly intended to steal someone's idea and "make it my own" by automatable transformation.
In the former case it seems to be a "genuine labour". (whatever that means in 2023)
The genuine artist intends to make something through intellectual labour.
Maybe that's the real question. I mean, about "labour". If the cost of the labour tends to zero, does it really matter?
Here's my take from an audio DSP viewpoint.
A composer listens to 10,000 hours of music. One day she writes an original piece based on the annealed parameters (temporal, spectral, intensity, pitch, sequence...) of a million artists. However it sounds like another specific artist who sues.
It is not a cover, a remix, homage, or even forged in the genre... it's just accidentally a bit too like it. (compare: Banana Splits vs. Bob Marley - and - Huey Lewis vs. Ray Parker Jr. - which completely misses the real impact of "Pop Muzik" by M)
The question is, was she exposed to influencing materials incorporated without intent or was it plagiarism (intent to reproduce a derivative etc) ?
By contrast I can take a piece of music, break it down by analysis into melodies, chords, timings, and use FFT to extract the precise spectrums of instruments, feed those to a resynthesis engine that finds new synthesiser parameters to create an exact sound-alike and then deliberately recreate a piece "In the near style of artist X". With a little musical processing I can change the key, inversions, re-template the rhythm to a new swing... always pushing the derived piece into new territory until eventually it's barely recognisable.
Nonetheless, in the second case I have clearly intended to steal someone's idea and "make it my own" by automatable transformation.
In the former case it seems to be a "genuine labour". (whatever that means in 2023)
The genuine artist intends to make something through intellectual labour.
Maybe that's the real question. I mean, about "labour". If the cost of the labour tends to zero, does it really matter?
The model isn’t sampling from training data during inference. It’s sampling from a latent space created during training, which used all of the training data to create it.
Fundamentally, the presence of a next token in some of the samples in the corpus is just as informative as the absence of that token from other samples. You can’t cite one without the other.
If you wanted to give attribution you’d need to list the entire training corpus.
Fundamentally, the presence of a next token in some of the samples in the corpus is just as informative as the absence of that token from other samples. You can’t cite one without the other.
If you wanted to give attribution you’d need to list the entire training corpus.
The failure mode I see here, which seems quite likely, is that attribution would almost always list tens of thousands of source files, or more. They doesn’t seem particularly useful or meaningful.
Yeah, correct, I would expect the same thing to happen if you list all attributions. That’s why I mentioned a criteria for filtering by number of tokens in a row for a given attribution, and one possible method for doing the filtering via largest common subsets.
While the very long list would be impractical (but not infeasible), sorting and filtering the list by how much each source contributes would make it easier to manage. Pruning all the entries that are below a contribution threshold would almost certainly shrink it to a very small fraction of the total number of attributions. We don’t need to list all the attributions that contribute to only one token, all we need to know is which attributions ended up with 5 or 50 or 500 tokens in a row, right? Likely not many.
While the very long list would be impractical (but not infeasible), sorting and filtering the list by how much each source contributes would make it easier to manage. Pruning all the entries that are below a contribution threshold would almost certainly shrink it to a very small fraction of the total number of attributions. We don’t need to list all the attributions that contribute to only one token, all we need to know is which attributions ended up with 5 or 50 or 500 tokens in a row, right? Likely not many.
> It’s not emitting “code”, it’s emitting individual tokens ... The “code” is the arrangement of those tokens, but that is determined by the weighting of the whole network
This theory of operation is not borne out in reality. It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input.
It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images.
And so, what can be traced back to individual sources? Quite a bit it would seem.
This theory of operation is not borne out in reality. It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input.
It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images.
And so, what can be traced back to individual sources? Quite a bit it would seem.
Well yes, if you’re asked to memorize someone’s code, you’ll get surprisingly far too. The fact that models can do this isn’t evidence of anything. It’s a capability (or “tendency” if you overfit too much).
I think it’s pretty obvious that if you train a model to reproduce existing work in its entirety, it fails the “sufficiently transformative” test and thus loses legal protection.
But there’s nothing stopping you from re-coding existing implementations of GPL’ed code. I used to do it. And your new code has your own chosen license, even if your ideas came from someone else. Are you sure the same logic shouldn’t apply to models?
I think it’s pretty obvious that if you train a model to reproduce existing work in its entirety, it fails the “sufficiently transformative” test and thus loses legal protection.
But there’s nothing stopping you from re-coding existing implementations of GPL’ed code. I used to do it. And your new code has your own chosen license, even if your ideas came from someone else. Are you sure the same logic shouldn’t apply to models?
> But there’s nothing stopping you from re-coding existing implementations of GPL’ed code.
There is if you've read the code. You can still violate copyright if you're hand-copying an implementation of a feature.
Lawyers recommend "clean room" re-implementations when reproducing copyrighted functionality (when folks bother to ask) precisely to avoid the risk of code being copied verbatim.
Functionality can be copied. Code can not.
There is if you've read the code. You can still violate copyright if you're hand-copying an implementation of a feature.
Lawyers recommend "clean room" re-implementations when reproducing copyrighted functionality (when folks bother to ask) precisely to avoid the risk of code being copied verbatim.
Functionality can be copied. Code can not.
Are you sure you can do that? Because it is trivial to paraphrase code and text.
Sure. I’ve done it for Bel.
EDIT: as far as I can tell, the only diff is that all comes before some. https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613&
I could have kept going through most of the implementation.
(def no (x)
(id x nil))
(def atom (x)
(no (id (type x) ‘pair)))
(def some (x f)
(if (no x) nil
(f (car x)) x
(some (cdr x))))
(def all (x)
(if (no x) t
(f (car x)) (all (cdr x))
nil)))
EDIT: as far as I can tell, the only diff is that all comes before some. https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613&
I could have kept going through most of the implementation.
I'd wager that most folks would consider the code for some basic list operations to be a little different from outputting a complete specific implementation of a fast square root function with its exact constants, including the comments.
Why? They’re both basic functions. I’d argue that if pg had left comments and constants, I’d have replicated those too.
Perhaps my example isn’t as impressive as the model’s capability, but operationally it’s the same.
Perhaps my example isn’t as impressive as the model’s capability, but operationally it’s the same.
Even if that theory of operation were true, it seems like an implementation detail that would be largely irrelevant in a court of law. Infringement is infringement regardless of how complicated your method for committing it was. If your AI reproduces a copyrighted work verbatim without permission, surely you are committing infringement? If you would be infringing by doing it via copy and paste, I would think a court would be likely to find that you infringed when you used an AI to do it. It does not matter that the algorithm on the backend is fancy...
Personally, I agree with you. Others do not.
But irrespective of legality, my point is that attribution is not such a hard problem as people believe, because their mental model of what's going on is much more complicated than reality. I feel they are seeing it as a true AI (mostly non-deterministic and thus hard to observe) and not ML which is deterministic.
But irrespective of legality, my point is that attribution is not such a hard problem as people believe, because their mental model of what's going on is much more complicated than reality. I feel they are seeing it as a true AI (mostly non-deterministic and thus hard to observe) and not ML which is deterministic.
> It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input.
Which makes sense when you consider that the sort of code that is getting reproduced verbatim is usually library functions which developers may copy and paste verbatim comments and all into their project, especially when you prompt the AI with the header of a function that has been copied and pasted often, so the weightings will in that instance be heavily skewed towards reproducing that function
Which makes sense when you consider that the sort of code that is getting reproduced verbatim is usually library functions which developers may copy and paste verbatim comments and all into their project, especially when you prompt the AI with the header of a function that has been copied and pasted often, so the weightings will in that instance be heavily skewed towards reproducing that function
So that should make it easy to attribute, yes?
I think harder, as it is spammed around in all directions. It's easier to attribute a unique piece of code that appears in a single repo.
But boilerplate functions don't deserve copyright protection as they are not creative. Can I copyright print('hello world!') if I post it in my repo? Do I deserve a citation from now on?
But boilerplate functions don't deserve copyright protection as they are not creative. Can I copyright print('hello world!') if I post it in my repo? Do I deserve a citation from now on?
For better or worse, AI is a combination of machine learning algorithms. And these algorithms are black boxes solely because we don't add observability to them - we aren't looking.
But there is a desire to understand why an AI provided the output it did (to increase trust in AI generated output), and so there's a lot of study and work going into adding that observability. Once that's in place, it becomes pretty straightforward to identify which inputs to a model provided what outputs.
But there is a desire to understand why an AI provided the output it did (to increase trust in AI generated output), and so there's a lot of study and work going into adding that observability. Once that's in place, it becomes pretty straightforward to identify which inputs to a model provided what outputs.
I have never seen an ML researcher claim that understanding the effect of specific training inputs on outputs is straightforward given the size of these LLMs. Most view it as a very difficult if not impossible problem.
And yet it's a major part of the overall concept of being responsible with our use of AIs. Throwing our hands up in the air and prematurely declaring defeat is not an option long term.
It's a non-starter for no other reason than potential copyright infringement means the government becomes involved, and they will stomp on the AI mouse with the force of an elephant - the opinions of amateurs and the anti-copyright movement notwithstanding.
As such, AI Observability is a problem that's both under active research, and the basis for B2B companies.
https://censius.ai/wiki/ai-observability
https://towardsdatascience.com/what-is-ml-observability-29e8...
https://whylabs.ai/observability
https://arize.com/
It's a non-starter for no other reason than potential copyright infringement means the government becomes involved, and they will stomp on the AI mouse with the force of an elephant - the opinions of amateurs and the anti-copyright movement notwithstanding.
As such, AI Observability is a problem that's both under active research, and the basis for B2B companies.
https://censius.ai/wiki/ai-observability
https://towardsdatascience.com/what-is-ml-observability-29e8...
https://whylabs.ai/observability
https://arize.com/
Observability is great but it doesn’t give granular enough insights into what is actually happening.
Given a black box you can do two things: watch the black box for a while to see what it does, or take it apart to see how it works.
Observability is the former. Useful in many cases, just not here.
If you want to know what LLMs are actually doing, you’ll need the latter. Looking at weight activations for example, although with billions of parameters that’s infeasible.
Given a black box you can do two things: watch the black box for a while to see what it does, or take it apart to see how it works.
Observability is the former. Useful in many cases, just not here.
If you want to know what LLMs are actually doing, you’ll need the latter. Looking at weight activations for example, although with billions of parameters that’s infeasible.
Those companies are not solving the problem you are describing
Probably why, like the article says, they're planning to add that
> In an attempt to address the issues with open-source licensing, GitHub plans to introduce a new Copilot feature that will “provide a reference for suggestions that resemble public code on GitHub so that you can make a more informed decision about whether and how to use that code,” including “providing attribution where appropriate.” GitHub also has a configurable filter to block suggestions matching public code.
> In an attempt to address the issues with open-source licensing, GitHub plans to introduce a new Copilot feature that will “provide a reference for suggestions that resemble public code on GitHub so that you can make a more informed decision about whether and how to use that code,” including “providing attribution where appropriate.” GitHub also has a configurable filter to block suggestions matching public code.
The output may be recognizable by humans as being “the same”, but an ML classifier can look at what is “obviously” a cat and classify it as a turtle. So while your statement is accurate, it doesn’t explain how “traceability” could really work—what is the actual mechanism of “tracing”?
There's a whole field of work around AI Observability - tracing how data moves through a model. It's intended to help identify biases in the input data, and to help answer the question of "why did it make this decision".
And if you can trace how data moves through the model, you can identify "that output came from these inputs" and add the associated metadata.
And if you can trace how data moves through the model, you can identify "that output came from these inputs" and add the associated metadata.
On your image generation point, something potentially interesting I saw on twitter/Reddit was about artists uploading “no to AI” images to artstation and then that logo almost appearing to vandalise AI generated art. I’m not sure if it’s a real thing or just people on twitter and stuff who don’t like AI art trolling though. Mostly because I’m not sure how up to date some of these models are being kept.
That's a joke/fake
> It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images.
Can you cite sources? I’ve heard this claim repeatedly but have yet to see a good example.
Can you cite sources? I’ve heard this claim repeatedly but have yet to see a good example.
https://youtu.be/kqPKNksl9hk?t=80 <- NatGeo cover
https://news.ycombinator.com/item?id=33061707 <- Watermarks (there's been a few linked here on HN)
https://news.ycombinator.com/item?id=33061707 <- Watermarks (there's been a few linked here on HN)
The AI learns to generate watermark-like things, because those exist on a large fraction of its inputs.
It doesn't mean the rest of the output existed in the training set. It's perfectly capable of generating a completely novel picture, then slapping a watermark on it.
It doesn't mean the rest of the output existed in the training set. It's perfectly capable of generating a completely novel picture, then slapping a watermark on it.
My point is that the watermark is deterministic. It is taken whole-cloth from an input, and reproduced as-is on an output. Thus, it is able to be attributed.
Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
> It is taken whole-cloth from an input, and reproduced as-is on an output.
I respectfully disagree with this claim. It’s not as-is. It’s remarkably similar. That’s a big difference.
> Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
I also disagree with this. The uncomfortable truth, imho, is that what Stable Diffusion does is FAR closer to what human artists do than we’d like to admit.
Human artists are perfectly capable of reproducing copyright infringing images. They just generally choose not to for legal/moral reasons.
I respectfully disagree with this claim. It’s not as-is. It’s remarkably similar. That’s a big difference.
> Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
I also disagree with this. The uncomfortable truth, imho, is that what Stable Diffusion does is FAR closer to what human artists do than we’d like to admit.
Human artists are perfectly capable of reproducing copyright infringing images. They just generally choose not to for legal/moral reasons.
> It is taken whole-cloth from an input, and reproduced as-is on an output
This is only possible for elements that are repeated hundreds or thousands of times. It's generally considered a failure of training data curation, if it's anything more than watermarks. The AI can learn how to reproduce the watermark, while being literally unable to reproduce the pictures it was on.
This is only possible for elements that are repeated hundreds or thousands of times. It's generally considered a failure of training data curation, if it's anything more than watermarks. The AI can learn how to reproduce the watermark, while being literally unable to reproduce the pictures it was on.
Neither of those are verbatim copies, do you have an example of a verbatim (i.e. exact, non-transformative) replica?
The quote does not make that assertion…
Do you have a source for this claim? This isn't really how generative models work
https://news.ycombinator.com/item?id=34187038 points to the two specific claims WRT stable diffusion. The co-pilot ones are easily found. Quake 3's "fast inverse square root" is the best known instance.
This quote seems to fundamentally misunderstand what transformers are doing at all. Technically I suppose you could save all gradient updates from every input token, and do some weighted averaging to show what inputs affected the particular output the most, but saving all those gradient updates would be unimaginably space consuming. "Feasible" is doing a lot of work there.
It's very hard for people to get away from the idea that GPT is "copying" something, but that's not what it's doing. The reality is, to get the exact artifact which produced the code in question, you need "Call me Ishmael" from Moby Dick just as much as the Linux kernel source.
It's very hard for people to get away from the idea that GPT is "copying" something, but that's not what it's doing. The reality is, to get the exact artifact which produced the code in question, you need "Call me Ishmael" from Moby Dick just as much as the Linux kernel source.
> The reality is, to get the exact artifact which produced the code in question, you need "Call me Ishmael" from Moby Dick just as much as the Linux kernel source.
Not always. Sometimes it just copies code without modification.
It never tells you when it does that, though. So to be on the safe side, better assume that it always does.
Not always. Sometimes it just copies code without modification.
It never tells you when it does that, though. So to be on the safe side, better assume that it always does.
Even then, the presence of a next token is just as informative as the absence of another during training. That information also gets backpropagated. And during inference, good luck identifying which weights were responsible for a given next token (assuming we’re using greedy decoding, don’t even get started on beam search) let alone which samples contributed to that weight (hint: they all did).
It sounds like nonsense. The most plausible solution would be to provide credit to every single author whose code was used in the original training set; of course, that would run into gigabytes just for the credits.
But GPL requires the resulting derivative code to be open-sourced under GPL, not just attributed. So if you did that then copilot could only emit code under a GPL license. Maybe that is okay, but it is not what copilot is trying to do at the moment. (And not everyone wants to open-source all their code under GPL.)
And…? I think my last Xcode update was 20+ Gb.
Personally, I don't want minimalistic programs to start taking 20+GB! Xcode's bloat isn't a goal to strive for.
If such a giant list of acknowledgements were to be included, would it even help anyone, in any way? It sounds like pure inefficiency.
If such a giant list of acknowledgements were to be included, would it even help anyone, in any way? It sounds like pure inefficiency.
[deleted]
In the ideal case the next token is determined by the local context (the prefix string) and the entire corpus of trained code. In this case the prefix string has not been seen before and so the generator must do some interpretation/extrapolation to determine the likely continuation. But in some cases, perhaps many cases, the prefix string has been seen before, or is similar enough to what has been seen before, that the best continuation is just to spit out the similar string in the training corpus. Presumably such cases can be detected due to specific patterns of activation in the network and attributions can be captured/applied.
One dumb way to do this would be to include self-attributions directly in the stream of training data. So in the cases where the best continuation is to just transcribe the training data, the attributions is included in the data itself.
One dumb way to do this would be to include self-attributions directly in the stream of training data. So in the cases where the best continuation is to just transcribe the training data, the attributions is included in the data itself.
That “presumably” is exactly what I’m trying to get past. How would it actually work? Once the input corpus has been chewed up and reduced to tokens, is there actually a representation of “the similar string in the corpus” any more?
I don’t think people are even clear on the problem statement here. If I fed in three very similar functions from different sources, and I got a fourth, also very similar, output, what is the “traced attribution” supposed to be?
I don’t think people are even clear on the problem statement here. If I fed in three very similar functions from different sources, and I got a fourth, also very similar, output, what is the “traced attribution” supposed to be?
There's a clear difference between an output string that matches 10's of tokens from training data and one that is synthesized from the statistical regularities of the training corpus as a whole and so will not match any input data past a few highly informative tokens (ie ignoring structural tokens). The computational dynamic of doing synthesis should be noticeably different from doing verbatim copying. For one, the statistically relevant context for doing synthesis is much larger than doing verbatim copying. The prior 5 tokens may be sufficient to determine that the continuation should be the verbatim copy (eg quake's fast inverse square root). This difference must be represented in the network activation patterns in some form or another. I don't know what this difference is, but I know it's there.
The case of genuine synthesis is the best case for code generators and aren't the cases where people are expecting attribution.
The case of genuine synthesis is the best case for code generators and aren't the cases where people are expecting attribution.
I see, so the hypothesis is there's some way of determining "how much choice" it has for each output token. The less choice it has, the more we will call what it's doing "verbatim copying". Once the threshold is crossed and you want to "trace" it, though, I'm still not clear on how you can map back through the latent space and figure out where in the corpus it's copying from. Other than the technique suggested upthread, where you just literally search the corpus as a separate operation.
Honestly, I would be fine with verbatim chunks of code if it could gaurantee a compatible license and did the copyright notices properly.
That always seemed like a problem with open source, you should theoretically be able to copy bits of code from hundreds of projects to make a new one, but keeping track of the licenses makes it too much of a pain. So the closest we really see is people vendorizing libraries.
That always seemed like a problem with open source, you should theoretically be able to copy bits of code from hundreds of projects to make a new one, but keeping track of the licenses makes it too much of a pain. So the closest we really see is people vendorizing libraries.
You can just find the most matching snippets in the training data using an embedding model.
They probably mean using a code search engine to check all snippets. The simplest thing would be am n-gram filter. A more advanced approach would use a code similarity neural net. It's not principled attribution, just locating the most similar example in the training set.
> It's not principled attribution, just locating the most similar
example in the training set.
This is a profoundly important distinction.
Back-tracing data to contributory training examples is a genuine "influenced by" relation. Picking the nearest neighbour to a given result (even if its an exact copy!) cannot say anything useful with respect to origins. And given that there will always be some proximate neighbour, it's really a "misattribution machine".
This is bit like how our broken patent system grants or denies ownership of a design based on similarity to extant art but regardless of actual originality.
This is a profoundly important distinction.
Back-tracing data to contributory training examples is a genuine "influenced by" relation. Picking the nearest neighbour to a given result (even if its an exact copy!) cannot say anything useful with respect to origins. And given that there will always be some proximate neighbour, it's really a "misattribution machine".
This is bit like how our broken patent system grants or denies ownership of a design based on similarity to extant art but regardless of actual originality.
So there a few ways this can actually be done with different levels of accuracy/precision. But it is going to be complicated no matter what.
The easy thing to do is to compare outputs to inputs. This isn't technically hard (i.e. cosine similarity) but it is computationally difficult (i.e. cosine similarity of output to entire input). This would give us some weightings that show similarity. But this doesn't really tell us attribution, rather more a correlation. These are statistical models so there's reason to believe that this is okay.
Then there's inversion. This is processing data backwards through the model. This has different complexities compared to what type of model you're working with. GANs aren't great, diffusions are okay, normalizing flows are trivial (but good luck generating good images from NFs). If we can invert the model it is much easier to investigate and probe for contribution by looking at the distance of the latent generative variable to the location of the latent trained data. Basically you're looking at how different information is contributing to the overall output. This can also be done at every level in the network. Obviously this gets both technically challenging as well as computationally.
Another method would be using dataset reconstruction (this is outside my wheelhouse fwiw). This is where you try to recover the dataset from the final trained network. This too is complicated but there's plenty of papers showing progress in this space (lots of interest from privacy groups).
(TLDR-ish) There's other methods too. But basically what is being said is that there are ways to denote what and how much the training data contributed to the output of the model wherein we can then measure how similar the output is to the inputs (i.e. copying).
The easy thing to do is to compare outputs to inputs. This isn't technically hard (i.e. cosine similarity) but it is computationally difficult (i.e. cosine similarity of output to entire input). This would give us some weightings that show similarity. But this doesn't really tell us attribution, rather more a correlation. These are statistical models so there's reason to believe that this is okay.
Then there's inversion. This is processing data backwards through the model. This has different complexities compared to what type of model you're working with. GANs aren't great, diffusions are okay, normalizing flows are trivial (but good luck generating good images from NFs). If we can invert the model it is much easier to investigate and probe for contribution by looking at the distance of the latent generative variable to the location of the latent trained data. Basically you're looking at how different information is contributing to the overall output. This can also be done at every level in the network. Obviously this gets both technically challenging as well as computationally.
Another method would be using dataset reconstruction (this is outside my wheelhouse fwiw). This is where you try to recover the dataset from the final trained network. This too is complicated but there's plenty of papers showing progress in this space (lots of interest from privacy groups).
(TLDR-ish) There's other methods too. But basically what is being said is that there are ways to denote what and how much the training data contributed to the output of the model wherein we can then measure how similar the output is to the inputs (i.e. copying).
The answer is of course it's possible, so long as you have a couple of
gigabytes spare for the acknowledgements page. Any construct worth
attributing will have roots in billions of parameters.
It's far worse than that: to attribute a particular output to an input you don't just need the input data, you need the gradient updates that data caused in the training run. A couple hundred gigabytes input tokens times 175 billion parameters equals... impossibility.
Theoretically possibly, technically challenging. The training data would need to be annotated with traces, so a person would have to figure out where all of the answers came from
Maybe like a science paper... 10 lines of code, 50 citations :)
To me AI code generators are the equivalent of crypto tumblers or mixers for digital coins. You can pretend all you want that the output is 'clean' but we all know it came from somewhere else and wasn't actually generated by the software, just endless little snippets that other people made.
This is definitely not true. In theory a generator could reproduce a distribution of data that includes more than the data it was able to sample from. For example, you could create the distribution of all possible faces from a subset of sample faces. That means you could create the faces of people who do not yet exist. Now good luck getting that perfect generator (black swans, bias, etc), but it would be naive to believe that they (well... that there can be no generator that) are just copy input data and mashing them together. At least if you're going to believe that that also isn't happening in the real world, but then what's the difference?
I don't think that's entirely true. When I've tested codepilot it also knows something about the context in which it's working, so it can use relevant variable names in it's suggestions. To me that's a step beyond rote regurgitation of other code.
That may be so, but the process is automated and copyright law has something to say about mechanical transformation.
As an example: if I take a piece of code you wrote and substitute all of the identifiers that does not create an original work.
As an example: if I take a piece of code you wrote and substitute all of the identifiers that does not create an original work.
ChatGPT output is very clean and precise especially when variables provided in detail. I doubt you can trace it back if you make the prompt very elaborate.
Why can't you trace it back? Remove all the input, fill the model with noise. Prompt all you want it will produce just noise. Train the model on your chosen input text. Suddenly it starts to provide 'clean and precise text'. Absent an argument that we're looking at AGI I see no other possible conclusion than that this is a mechanical transformation of two inputs (yours + the training data) into some output. What happens inside the box is irrelevant, that's just implementation details of the mechanism. If you build your mechanism in such a way that you can't trace it yourself that doesn't mean that you get to claim the output as an original work.
What kind of platform can detect what variables or instructions I added? It can be very difficult to reverse engineer the prompt from the output.
That's not the problem though. The onus would be on you to prove that your machine generated the output and that it wasn't based on the training data but just on your input. Otherwise the generated work isn't yours.
That's the moral aspect of the problem. First thing that I am concerned is that the search engines will be flooded by the bots creating ai content and engines won't have a way to identify it.
Sure, but that's not a copyright issue, that's just the problem of the web in general: by measuring something (ranking the importance of a page) you've introduced an element that can be gamed and whatever means people can use to game it they will use.
But copyright is an entirely different kettle of fish and it is very well possible that Microsoft/GitHub will be able to do an end run around copyright but I don't think they've really thought through what the consequences of that will be. Essentially they are saying 'if you have access to a bunch of source code you are free to slap new terms and conditions on the hosting of that code and then you can use it to train your models, even without active consent of the copyright holders'. In that world Microsoft stands a lot to lose. Much more than your average GitHub repository owner. So this was a terrible mistake on their part.
But copyright is an entirely different kettle of fish and it is very well possible that Microsoft/GitHub will be able to do an end run around copyright but I don't think they've really thought through what the consequences of that will be. Essentially they are saying 'if you have access to a bunch of source code you are free to slap new terms and conditions on the hosting of that code and then you can use it to train your models, even without active consent of the copyright holders'. In that world Microsoft stands a lot to lose. Much more than your average GitHub repository owner. So this was a terrible mistake on their part.
[deleted]
I propose that information naturally wants to degrade. Paper decomposes. Bits flip. File formats are replaced and lost. Storage mediums degrade. It's all an extension of the universe trending towards entropy.
It actually takes quite a bit of effort to store, then distribute information precisely and broadly. There's a lot of infrastructure, effort, and money involved, and still information degrades and disappears over time.
Any libre information exists because people have put effort into it. Sometimes a lot of effort.
It actually takes quite a bit of effort to store, then distribute information precisely and broadly. There's a lot of infrastructure, effort, and money involved, and still information degrades and disappears over time.
Any libre information exists because people have put effort into it. Sometimes a lot of effort.
I cede your larger point that information availability requires upkeep, but I don’t think the tendency for information to degrade is usually attributable to entropy. It’s probably true enough in some cases, but more often than not it’s going to be general interest by the population that determines the accessibility of information—and that crosses boundaries and regulatory efforts like copyright. The more intense the interest and/or awareness (generally speaking) the more likely the information will find its way into the public domain.
It’s why you can easily find literature written hundreds or thousands of years ago in multiple languages with little effort, but some memo stored in a tape archive from the 1970s is likely gone forever. “Bitrot” is more likely to be a function of diminishing interest rather than increasing entropy.
It’s why you can easily find literature written hundreds or thousands of years ago in multiple languages with little effort, but some memo stored in a tape archive from the 1970s is likely gone forever. “Bitrot” is more likely to be a function of diminishing interest rather than increasing entropy.
> It’s why you can easily find literature written hundreds or thousands of years ago in multiple languages with little effort
Little effort, because someone else put in the effort to keep and disseminate it. And we only know of the surviving material, we have no idea how much was lost.
Take the bible for example. It's a classic example of stories from millennia ago making it to the modern world. But its journey took the concerted effort of millions of scribes, translators, and so forth to make the journey. And it has not done so intact. We can even see how the Bible's been changed by comparing just the last two major revisions of it: NIV and KJ. Language aside, there are some pretty major ideological changes based on interpretation. And thanks to some miraculous archeological finds in the dead sea scrolls, we can see how the bible has drifted even further from its original roots.
That's not information being free, it's information degrading and being re-purposed - often re-created - for specific ideologies and beliefs.
Little effort, because someone else put in the effort to keep and disseminate it. And we only know of the surviving material, we have no idea how much was lost.
Take the bible for example. It's a classic example of stories from millennia ago making it to the modern world. But its journey took the concerted effort of millions of scribes, translators, and so forth to make the journey. And it has not done so intact. We can even see how the Bible's been changed by comparing just the last two major revisions of it: NIV and KJ. Language aside, there are some pretty major ideological changes based on interpretation. And thanks to some miraculous archeological finds in the dead sea scrolls, we can see how the bible has drifted even further from its original roots.
That's not information being free, it's information degrading and being re-purposed - often re-created - for specific ideologies and beliefs.
AI is inherently incompatible with the notion of private property as a result of the way it is "trained". It unintentionally exposes how much of our creativity, exploration, and innovation we've commodified.
That's why it wants to be free. To survive by being copied and adapt by being freely built upon.
considering that information is something static (which doesn't change) that describes (is about) how something else is changing
I think that indeed, it is the inherent nature of information to radiate itself, i.e. to share (to shine, to spread)
I think that indeed, it is the inherent nature of information to radiate itself, i.e. to share (to shine, to spread)
When Copilot was released the copyright discussion focused exclusively on code, but now we see very similar discussion around images with stable diffusion and the sister project unstable diffusion. When Copilot do reach the courts there will be some indication on how courts view author consent when it comes to training material. After that we might then see court cases for each form of media (images, video, text, sound), and also in their each unique contexts (books, online videos, porn, stock databases, and so on).
In my opinion using all of the code on GitHub without respecting the licenses was a capital mistake. It should have been opt-in, maybe with some incentive but to just take it all without so much as a by-your-leave is not going to play well in court.
In my opinion it was covered under the GitHub terms of service and is clearly transformative. I am optimistic that the courts will find it so and we can put these debates to rest similar to how we’ve done for web scraping.
Sorry but terms of service don't give you a blanket license to re-purpose someone else's copyrighted work at your pleasure. By that same token any hosting provider could make a small change to their terms of service and suddenly all of the data of all of the customers would be theirs. Copyright does not work that way, you need to actively sign away your rights.
> Sorry but terms of service don't give you a blanket license to re-purpose someone else's copyrighted work at your pleasure.
That’s correct. It’s also not what happened here. I don’t violate your copyright when I scrape your web page. I don’t violate your copyright when I train a model on a web page that I scraped.
I violate your copyright only if I use that tool to produce code that violates your copyright. This is similar to how I don’t commit a copyright violation when I read your code, I commit the violation when I produce and publish code that violates your copyright.
That’s correct. It’s also not what happened here. I don’t violate your copyright when I scrape your web page. I don’t violate your copyright when I train a model on a web page that I scraped.
I violate your copyright only if I use that tool to produce code that violates your copyright. This is similar to how I don’t commit a copyright violation when I read your code, I commit the violation when I produce and publish code that violates your copyright.
If the training data discussion ends on that note than there is exist a large upside. Youtube and other video hosting services has a massive collection of the worlds images, music and sound. Anyone who who want to build an AI will have access to practically unlimited amount of training data, meaning the barrier to entry will be quite low.
I doubt YouTube will let anyone download more than a trivial fraction of the videos they host, but it has been used for some specialized projects like Minecraft AI (70,000 hours of video).
https://openai.com/blog/vpt/
https://openai.com/blog/vpt/
First you wrote:
> In my opinion it was covered under the GitHub terms of service and is clearly transformative
Now you change the argument completely and let's just say that even with that change that is not how I understand that copyright works and leave it at that. You're very welcome to your own interpretation. Best of luck.
> In my opinion it was covered under the GitHub terms of service and is clearly transformative
Now you change the argument completely and let's just say that even with that change that is not how I understand that copyright works and leave it at that. You're very welcome to your own interpretation. Best of luck.
The GitHub ToS only applies a blanket license to copy as a fallback if you forget to stick a license file in your code - i.e. it's there so that if you post code to GitHub you can't sue GitHub for doing the thing you told it to do. Furthermore, that only covers stuff intentionally uploaded to GitHub by the author; people mirroring repos hosted elsewhere do not have the legal capacity to license attribution-free, copyleft-free AI training.
Apple made that interpretation regarding their app-store. It is also the reason why GPL software is not accepted on the store. Authors would need to give apple a full copyright transfer agreement (or exemptions of the GPL terms), including any dependencies, and that was simply more costly than just banning GPL from the store.
It would have been awesome if Copilot had been written so that you tell it what your license is and it emits code trained only on code with compatible licenses. That would have created an incentive for people to publish their own code under more open licenses, instead of facilitating IP theft and thus creating a disincentive. That GH didn't choose this option is rather telling IMO.
Yes, that would have been much, much better. Better yet if you could trace it back to the authors so that in the case of attribution licenses you could have properly attributed the code.
I think it’s silly to pretend that human programmers are emitting a lot of code with a high degree of originality either. We’re all remixing some long-forgotten influential code laying deep in latent memory just like the models.
Humans do remix stuff when we create, and copyright draws a kind of arbitrary line around when that becomes original and when it becomes derivative, mostly because if it didn't then it would be impossible for us to get new books, movies, and songs.
All copyright is a hack; it's not an ideological, internally consistent framework. It is a set of fallible legal rules we invented so that people would get paid for creating things we care about. That doesn't mean that comparisons aren't useful or that we can't extrapolate from the existing rules, but even where fair use is concerned, the actual justification isn't a logical one, it's: "if we didn't have this standard nobody would make anything." So the boundaries around fair use and what things fall into it are not being created purely based on logic or first principles.
The reason why we have copyright is because we want people to be paid for making creative work. The reason we have fair use and a standard of "originality" that treats coders/artists learning from other artists as acceptable is because if we didn't, the entire system would fall apart and nobody would be able to make new creative works.
Everything in copyright exists purely to get people to make more stuff in a sustainable way. It's an outcome-driven process.
----
More recently, there are a lot of people who argue that IP is a real, fundamental property right, but frankly, IP doesn't stand up at all if you think about it too hard. The justifications for why IP theft is theft can't be consistently generalized in a way that applies outside of the IP space. The standards for what does and doesn't count as creative aren't really consistent or based on a straightforward definition of creativity.
A lot of people would love to say that IP rights are just property rights, but... it's not all that convincing, and the history of copyright doesn't really indicate to me that the people building the laws thought of them that way.
And again, that doesn't mean that there's no consistency in copyright rules or that copyright rulings don't have implications beyond the original rulings. But it is almost always easier to think about copyright and almost always easier to understand why copyright laws are the way that they are if you approach copyright as a means to an end, and understand the existing laws not as an attempt to create an internally consistent system, but as a series of attempts throughout US history to achieve a consistent publicly beneficial outcome.
----
With that in mind, I suspect whether or not AI works count as remixing is largely going to be decided based on commercial interests, individual judges, and individual juries, possibly with input from US legislature.
"Everybody remixes" historically hasn't been the most useful argument during these debates? So I don't know how it's going to play out this time around. I vaguely suspect it's going to come down to whether or not individual pieces are recognizable? That's how we got wild copyright laws about some individual chord progressions being treated as derivative works in songs.
All copyright is a hack; it's not an ideological, internally consistent framework. It is a set of fallible legal rules we invented so that people would get paid for creating things we care about. That doesn't mean that comparisons aren't useful or that we can't extrapolate from the existing rules, but even where fair use is concerned, the actual justification isn't a logical one, it's: "if we didn't have this standard nobody would make anything." So the boundaries around fair use and what things fall into it are not being created purely based on logic or first principles.
The reason why we have copyright is because we want people to be paid for making creative work. The reason we have fair use and a standard of "originality" that treats coders/artists learning from other artists as acceptable is because if we didn't, the entire system would fall apart and nobody would be able to make new creative works.
Everything in copyright exists purely to get people to make more stuff in a sustainable way. It's an outcome-driven process.
----
More recently, there are a lot of people who argue that IP is a real, fundamental property right, but frankly, IP doesn't stand up at all if you think about it too hard. The justifications for why IP theft is theft can't be consistently generalized in a way that applies outside of the IP space. The standards for what does and doesn't count as creative aren't really consistent or based on a straightforward definition of creativity.
A lot of people would love to say that IP rights are just property rights, but... it's not all that convincing, and the history of copyright doesn't really indicate to me that the people building the laws thought of them that way.
And again, that doesn't mean that there's no consistency in copyright rules or that copyright rulings don't have implications beyond the original rulings. But it is almost always easier to think about copyright and almost always easier to understand why copyright laws are the way that they are if you approach copyright as a means to an end, and understand the existing laws not as an attempt to create an internally consistent system, but as a series of attempts throughout US history to achieve a consistent publicly beneficial outcome.
----
With that in mind, I suspect whether or not AI works count as remixing is largely going to be decided based on commercial interests, individual judges, and individual juries, possibly with input from US legislature.
"Everybody remixes" historically hasn't been the most useful argument during these debates? So I don't know how it's going to play out this time around. I vaguely suspect it's going to come down to whether or not individual pieces are recognizable? That's how we got wild copyright laws about some individual chord progressions being treated as derivative works in songs.
by reducing credit, copilot reduces incentive to create and publish free code. biting the hand that feeds it.
exact same problem exists with GPT3 and others.
big tech slashing and burning, ruthlessly exploiting the least empowered people in the tech economy.
neat hack.
exact same problem exists with GPT3 and others.
big tech slashing and burning, ruthlessly exploiting the least empowered people in the tech economy.
neat hack.
I used to put most of my code on public repos in Github, Gitlab, etc. Since Copilot came out, I've moved most of my projects to a Git server on my home network.
The prospect of knowledge work being resold in this way feels icky. Perhaps I'd feel different if there was a simple, well-defined way to opt out of contributing to the training corpus. Opt-in would be even better.
The prospect of knowledge work being resold in this way feels icky. Perhaps I'd feel different if there was a simple, well-defined way to opt out of contributing to the training corpus. Opt-in would be even better.
Copilot also makes it easier to use open source framework code.
Being able to ask a model for help writing Spark, SQLAlchemy, Tokio, whatever, actually increases the usability of free code vs proprietary code.
Being able to ask a model for help writing Spark, SQLAlchemy, Tokio, whatever, actually increases the usability of free code vs proprietary code.
How does copilot reduce the incentive to publish free code?
If it writes what would have been open source dependencies based on open source code you no longer have to license that work or your own. Also coders might not want to open source original work for fear they are just going to feed that beast that will ultimately kill them. Sort of a chilling effect if they cannot chose to contribute to that cause or not while also sharing code as open source.
Depends how you define free.
If an AI being inspired by GPL code that happened to be in its training data because someone other than the creator stuck it in a repo on GitHub is just fine, and if some code produced as a result is practically identical that is fine too, and the resulting code is not considered GPL any more, then the GPL and licences like it are worthless.
The only two designations that mean anything at that point are public domain and commercial. “free” as in “Free” rather than public domain means the same as public domain.
So there is no point releasing free (other than public domain) code, discouraging the act. If I want to control how my code is used at all protected commercial release becomes the only option.
This of course suits the commercial interest behind copilot just fine and dandy…
---
But, if commercial code ended up in the training set the same should apply because in terms of giving the right to use code licences like the GPL and commercial licences are no different: the licence gives the right to use the code. If passing it through an AI gives that right, bypassing the licence, for one case than it should for the other too. I wonder if MS would be happy for copilot's own code to be in the training set and for me to produce and sell something based on the output of an AI trained with their code?!
---
I think the AI systems like copilot should be considered the same as us wet-ware naturally formed intelligence systems in that respect: if it produces something based on code under a particular licence then that something should be subject to the terms of the licence. Ignorance of the licence is no excuse. If the AI can not be made aware of the correct licence and attribution for the code, so it can include that with its suggestions based on it, then that code should not be in the training set.
For decades MS complained about open source code potentially creating this very situation, just with only non-artificial intelligences in the mix, now they are hoping no one can call them on that because it is convenient for them to ignore the issue.
If an AI being inspired by GPL code that happened to be in its training data because someone other than the creator stuck it in a repo on GitHub is just fine, and if some code produced as a result is practically identical that is fine too, and the resulting code is not considered GPL any more, then the GPL and licences like it are worthless.
The only two designations that mean anything at that point are public domain and commercial. “free” as in “Free” rather than public domain means the same as public domain.
So there is no point releasing free (other than public domain) code, discouraging the act. If I want to control how my code is used at all protected commercial release becomes the only option.
This of course suits the commercial interest behind copilot just fine and dandy…
---
But, if commercial code ended up in the training set the same should apply because in terms of giving the right to use code licences like the GPL and commercial licences are no different: the licence gives the right to use the code. If passing it through an AI gives that right, bypassing the licence, for one case than it should for the other too. I wonder if MS would be happy for copilot's own code to be in the training set and for me to produce and sell something based on the output of an AI trained with their code?!
---
I think the AI systems like copilot should be considered the same as us wet-ware naturally formed intelligence systems in that respect: if it produces something based on code under a particular licence then that something should be subject to the terms of the licence. Ignorance of the licence is no excuse. If the AI can not be made aware of the correct licence and attribution for the code, so it can include that with its suggestions based on it, then that code should not be in the training set.
For decades MS complained about open source code potentially creating this very situation, just with only non-artificial intelligences in the mix, now they are hoping no one can call them on that because it is convenient for them to ignore the issue.
I have learned from a lot of commercial code. It doesn't mean I can't also write "free" code.
I find most things in all code to be generic and obvious. The same functions are used by lots of other software with just different names that there's nothing novel.
After you've learned to classify lots of functions with big O notation, you will end up writing things in an obvious way for a base-level optimization that fits naturally.
I find most things in all code to be generic and obvious. The same functions are used by lots of other software with just different names that there's nothing novel.
After you've learned to classify lots of functions with big O notation, you will end up writing things in an obvious way for a base-level optimization that fits naturally.
> by reducing credit
I would have thought that in the vast majority of current AI-generated code we are talking about single blocks and functions that are just Intellisense on steroids that only a rather self-deluding coder would consider original enough or "theirs" to attribute authorship to.
There are no doubt grey areas and more serious cases as the technology improves and the generated content increases in length and functional value, but I hope we don't throw the productive baby out with the Luddite bathwater...
There are no doubt grey areas and more serious cases as the technology improves and the generated content increases in length and functional value, but I hope we don't throw the productive baby out with the Luddite bathwater...
I don’t disagree with you but I think there’s another angle to the discussion that involves the difference between the litigious capacity of corporations vs individuals to defend the simple kind of creations you describe, as I think corporations have defended (and even patented) very simple patterns before.
For what it’s worth, I wish the approach was as you describe but in all cases: make it impossible to defend attribution of basic building blocks regardless of how deep your pockets are.
For what it’s worth, I wish the approach was as you describe but in all cases: make it impossible to defend attribution of basic building blocks regardless of how deep your pockets are.
I think that kind of depends on how common the snippet is. If it's code for swapping to variables, well everyone has written that and it's small. But what if it's a rarer thing like a new kind of debayering algorithm that takes a much larger "snippet" from a novel algorithm? As far as I'm aware there are no rigorous studies showing the level to which one could duplicate longer stretches of code and if it can do something like this.
Algorithms aren't copyrightable as far as my understanding goes and this is where it gets murky. The expression is copyrightable, the algorithm isn't (except when it is...)
My personal feeling is that we've gone too far down the road of assigning IP to code and we should be rolling it back. I'd hate for the current AI boom to trigger an extension of current IP law.
My personal feeling is that we've gone too far down the road of assigning IP to code and we should be rolling it back. I'd hate for the current AI boom to trigger an extension of current IP law.
The abstract algorithm isn't covered by copyright, but can be patented (lots of asterisks here about software parents). But the exact lines of code you write are covered by copyright.
If you tell Github Copilot to write a fast inverse square sum algorithm it's allowed to reproduce the idea of the famous quake3 algorithm, but if it produces the same lines and comments as those in the quake 3 source code then that's a pretty open and shut copyright violation
If you tell Github Copilot to write a fast inverse square sum algorithm it's allowed to reproduce the idea of the famous quake3 algorithm, but if it produces the same lines and comments as those in the quake 3 source code then that's a pretty open and shut copyright violation
> But the exact lines of code you write are covered by copyright.
I believe it's "only if they are expressive" i.e. code that is pretty much what anyone would have written in that situation isn't copyrightable.
I think this is the part where we should be much stricter in applying the principle. Similar to "novel and non-obvious" in patent law, it's become diluted in favour of IP overreach.
I believe it's "only if they are expressive" i.e. code that is pretty much what anyone would have written in that situation isn't copyrightable.
I think this is the part where we should be much stricter in applying the principle. Similar to "novel and non-obvious" in patent law, it's become diluted in favour of IP overreach.
Leaving aside what I think IP law ought to look like, LLMs literally train on (and reproduce!) the fixed, copyrighted expression. They aren't producing a new work based on abstract knowledge of algorithms that happens to share a particular expression.
It's not at all obvious to me why the training set copyright wouldn't nominally flow through the model, even if that seems impossible to actually implement in practice.
It's not at all obvious to me why the training set copyright wouldn't nominally flow through the model, even if that seems impossible to actually implement in practice.
Why is everyone fixating on 1% of the outputs. 99% are sufficiently distinct. It replicates code as an unintended side effect, but there are mitigations.
If you take all the words from Harry Potter and shuffle the words together in a random order, the output will be distinct but the work as a whole is probably still derivative. If you do that for 500 book series, you still haven't generated a new work, you've simply created a legal morass of derivative copyrights.
ChatGPT is just a really impressive word-blender. All of those "sufficiently distinct" snippets are in a meaningful sense recombinations of other copyrighted snippets it was trained on. When you're in a very sparse part of the space or have a single work that's present many times in training like fast inverse square root, the model may even replicate recognizable chunks of the training set. It could be that this is sufficiently transformative that the output is not derivative, but as I said it's not obvious to me.
ChatGPT is just a really impressive word-blender. All of those "sufficiently distinct" snippets are in a meaningful sense recombinations of other copyrighted snippets it was trained on. When you're in a very sparse part of the space or have a single work that's present many times in training like fast inverse square root, the model may even replicate recognizable chunks of the training set. It could be that this is sufficiently transformative that the output is not derivative, but as I said it's not obvious to me.
> If you take all the words from Harry Potter and shuffle the words together in a random order, the output will be distinct but the work as a whole is probably still derivative. If you do that for 500 book series, you still haven't generated a new work, you've simply created a legal morass of derivative copyrights.
This might be the worst metaphor I've stumbled across in this whole debate. You're surely making an argument here for the opposite team.
This might be the worst metaphor I've stumbled across in this whole debate. You're surely making an argument here for the opposite team.
Every corporate code base is copyrighted. Engineers can’t write code similar to what they worked on, why should a corporate AI be able to violate copyleft?
Yes, of course, but that isn't the "vast majority" of use cases I pointed out; most people will presumably be using it for bog standard stuff in place of googling and thinking for a short while - let alone the people who didn't need the memory jog and are just selecting from an automatic prompt to save them typing it.
You can generate whole linux kernel drivers with eg. ChatGPT. And things will only get better, so your dismissal is not very forward looking. The issue will need to be dealt with.
You should reread my comment to see that I explicitly made exceptions for more advanced cases.
I can only speak to my experience using ChatGPT. But it’s doing a lot more than copying and pasting code snippets it finds on the internet. It actually is translating English to code.
I had a “DevOps” project I was working on creating deployment process using AWS technologies (disclaimer: where I work in Professional Services). I needed a few relatively simple Python scripts.
I first asked ChatGPT:
“given a JSON file like this [{“company”:”${company}”} replace the word surrounded by ${} with the equivalent environment variables using Python”.
It worked perfectly. But it hardcoded the input file.
“Modify the script to accept the input file using a command line argument -json-file using argparse”
That worked and used the “required=true” parameter.
“Instead of skipping replacement if an environment variable is not found, raise an exception”.
And it seems to understand the AWS SDK. I told it to convert a script I wrote to a CloudFormation custom resource using the cfnresponse module and it knew the correct Lambda event structure, the event format etc.
I have seen reports and have witnessed it making up functions occasionally.
But, I believe with the right prompts, it should be able to create simple CRUD scripts.
I had a “DevOps” project I was working on creating deployment process using AWS technologies (disclaimer: where I work in Professional Services). I needed a few relatively simple Python scripts.
I first asked ChatGPT:
“given a JSON file like this [{“company”:”${company}”} replace the word surrounded by ${} with the equivalent environment variables using Python”.
It worked perfectly. But it hardcoded the input file.
“Modify the script to accept the input file using a command line argument -json-file using argparse”
That worked and used the “required=true” parameter.
“Instead of skipping replacement if an environment variable is not found, raise an exception”.
And it seems to understand the AWS SDK. I told it to convert a script I wrote to a CloudFormation custom resource using the cfnresponse module and it knew the correct Lambda event structure, the event format etc.
I have seen reports and have witnessed it making up functions occasionally.
But, I believe with the right prompts, it should be able to create simple CRUD scripts.
I understand what it is doing, and how impressive it is; the question is whether or not the normal use case is for stuff that requires the originality of thought that requires attribution, and I can't see that being the case except in a very small minority of instances.
I’m horrified.
I know we’re “just” talking about code here but decisions will be far-reaching and if we let the powers that be force AI-generated content to be copyright-attributed, two things are going to happen:
1. The biggest benefits of AI are going to be pushed back for decades and possibly indefinitely
2. Artists will be absolutely slaughtered as the same rules will come at them full-force. Almost every artist draws inspiration from other creative work. That’s how creativity works. Can you imagine how stifling it would be for every artist to have to document and “pay for” every piece of art they’ve ever seen??
I know we’re “just” talking about code here but decisions will be far-reaching and if we let the powers that be force AI-generated content to be copyright-attributed, two things are going to happen:
1. The biggest benefits of AI are going to be pushed back for decades and possibly indefinitely
2. Artists will be absolutely slaughtered as the same rules will come at them full-force. Almost every artist draws inspiration from other creative work. That’s how creativity works. Can you imagine how stifling it would be for every artist to have to document and “pay for” every piece of art they’ve ever seen??
As well, this:
https://news.ycombinator.com/item?id=34242124
If AI can produce patentable and copyrightable proteins, the first pharmaceutical company to press a button will own all possible proteins. And then all possible permutations of every undiscovered drug.
https://news.ycombinator.com/item?id=34242124
If AI can produce patentable and copyrightable proteins, the first pharmaceutical company to press a button will own all possible proteins. And then all possible permutations of every undiscovered drug.
I’m horrified.
If we let the powers that be force AI-generated content to be copyrighted, two things are going to happen:
1. The biggest benefits of AI are going to be pushed back for decades and possibly indefinitely
2. Artists will be absolutely slaughtered as the same rules will come at them full-force. Almost every artist draws inspiration from other creative work. That’s how creativity works. Can you imagine how stifling it would be for every artist to have to document and “pay for” every piece of art they’ve ever seen??
If we let the powers that be force AI-generated content to be copyrighted, two things are going to happen:
1. The biggest benefits of AI are going to be pushed back for decades and possibly indefinitely
2. Artists will be absolutely slaughtered as the same rules will come at them full-force. Almost every artist draws inspiration from other creative work. That’s how creativity works. Can you imagine how stifling it would be for every artist to have to document and “pay for” every piece of art they’ve ever seen??
It seems to me there are some really fundamental questions about copyright and use posed here that have been submerged in the background of the internet rush to digitize all forms of content and expression. Even though these generative tools may simplify the path to verbatim code (or image, or text) copying without attribution, and complicate determining what, if any, attributions are required, they didn't invent it. A browser coupled with a search engine facilitates such copying all the time. Like the generative models, the search engine is a tool that has read, for its own purposes, vast swaths of content, remembered key details about it, and serves a means to render the content up to a user. It's not the copying per se that causes trouble - creators make their material on the internet with the expectation and hope that it will be copied, by all those tools, into search engines and ultimately onto screens, so I can see it. It's that the chain of attribution is lost (or with OS code, the license imprint, which is really just another form of attribution), and with the attribution, any hope that the creator will benefit from the copying.
But actually carrying attribution forward is going to be hard. Things that come out of a generative model that are substantially identical to a particular input do so for one of two reasons: one particular training input to the model is overwhelmingly the "best" source of response to the prompt, or a training input has been repeated over and over again in the training set so as to become the consensus response to one or more prompts. The first might reasonably feasible to track down, although it's bound to be computationally expensive. The second ... really tough, since it forces the model to "know" which of those many training sources is the one that should attributed. The widely circulated example of an image generation model reproducing Steve McMurray's famous photo of Sharbat Gula, a green-eyed Afghani girl that appeared on National Geographics cover in 1985 shows the problem. Do a Google search for "green-eyed Afghan girl" and you'll find hundreds of copies of varying resolution and definition, and hundreds more derivative versions, of McMurray's photo. A model spitting out yet another derived, but nearly identical version is likely drawing from hundreds of those images itself, not some original root, copyrighted golden copy. Which should it attribute?
But actually carrying attribution forward is going to be hard. Things that come out of a generative model that are substantially identical to a particular input do so for one of two reasons: one particular training input to the model is overwhelmingly the "best" source of response to the prompt, or a training input has been repeated over and over again in the training set so as to become the consensus response to one or more prompts. The first might reasonably feasible to track down, although it's bound to be computationally expensive. The second ... really tough, since it forces the model to "know" which of those many training sources is the one that should attributed. The widely circulated example of an image generation model reproducing Steve McMurray's famous photo of Sharbat Gula, a green-eyed Afghani girl that appeared on National Geographics cover in 1985 shows the problem. Do a Google search for "green-eyed Afghan girl" and you'll find hundreds of copies of varying resolution and definition, and hundreds more derivative versions, of McMurray's photo. A model spitting out yet another derived, but nearly identical version is likely drawing from hundreds of those images itself, not some original root, copyrighted golden copy. Which should it attribute?
Why have so many of these AI models been trained on GPL licensed code? Almost half the controversy could have avoided by ignoring GPL code.
I know that even non-copyleft licenses like the Apache and MIT licenses are copyrighted and require attribution however it would have caused far less controversy than training on GPL licensed code.
I know that even non-copyleft licenses like the Apache and MIT licenses are copyrighted and require attribution however it would have caused far less controversy than training on GPL licensed code.
Yes, whoever signed off on that wasn't thinking clearly.
Code cannot be owned. A creative expression may be copyrighted. Purely functional expressions may not be copyrighted. The output of a trained AI is insufficiently creative to be copyrighted. Only humans can hold a copyright.
Now with all that, there really isn’t anything here to get worked up over.
Now with all that, there really isn’t anything here to get worked up over.
"Purely functional expressions may not be copyrighted."
Is this more than a opinion?
Because you can have whole programs as a long functional expression (not that I am a fan of such a coding style, but it exists).
Is this more than a opinion?
Because you can have whole programs as a long functional expression (not that I am a fan of such a coding style, but it exists).
It is an accurate statement of the law in the US, but "purely functional expression" is not used in the sense of a mathematical function. It basically means expressions that solely reflect the way you need to do the thing to accomplish the function you are trying to achieve, without any significant creative element.
For example, if I asked you to write Java code to print the sum of two integer variables named a and b, followed by a newline, you'd probably write pretty much exactly this:
System.out.println(a + b);
You might vary the spacing around the punctuation, or a few other unimportant details, but there's not much room for creativity there while still accomplishing the desired function. It isn't copyrightable.
And this is a good example of how the word functional means something non-mathematical here: that isn't even a functional expression in the mathematical sense, since there's no return value and the printing is a side effect. It's purely imperative.
For example, if I asked you to write Java code to print the sum of two integer variables named a and b, followed by a newline, you'd probably write pretty much exactly this:
System.out.println(a + b);
You might vary the spacing around the punctuation, or a few other unimportant details, but there's not much room for creativity there while still accomplishing the desired function. It isn't copyrightable.
And this is a good example of how the word functional means something non-mathematical here: that isn't even a functional expression in the mathematical sense, since there's no return value and the printing is a side effect. It's purely imperative.
It seems like a very blurry line for any non-trivial piece of code.
It is a blurry line, like many legal tests. Human judges and/or juries apply this test in case of litigation, as advised by whatever experts the parties are able to to offer.
> It seems like a very blurry line
Opinion me, this statement could apply to the vast majority of copyright law in the US.
Opinion me, this statement could apply to the vast majority of copyright law in the US.
Haha, imagine they make all the noise now and get language models to solve attribution, but as a side effect all human plagiarism that is hiding in plain sight will be revealed as well. Be careful what you wish for.
It's not worded great (code can be owned under current law). But surprisingly enough, there is a core here that is more than opinion, it just needs some elaboration.
It's also going to be hotly debated in the future, because right now most of the commercial AI-generators are just kind of ignoring this and at some point I think they're going to make the argument, "this law/interpretation can't hold or else it would be commercially devastating for us, and it needs to change."
But under current US copyright law, the US copyright office has pretty consistently ruled that:
- purely functional expressions without a creative aspect can't be copyrighted (ie, you patent an invention, you don't copyright it, and stuff like recipes aren't eligible for copyright at all, only the surrounding text is).
- AI-generated content does not have a creative aspect and isn't eligible for copyright protection.
They've even gone so far as to revoke copyright protections to AI-generated content[0][1]. We haven't gotten a similar ruling about code that I'm aware of, but given that the US copyright office already only grants copyright to code under the assumption that coding is a creative act, it's very difficult to imagine code having more legal protections than an image or a book.
----
In my opinion, this is the much more interesting debate than model's source[2], because nobody is really talking about it, and depending on how the inevitable legal challenges play out, it could completely change the field. It's an interesting choice to think about -- let's say an AI gets good enough that you can sit down, describe an app, and the AI just generates it straight up. Would you still use that tool even if the resulting code was public domain and you couldn't assert ownership over it?
Which would be more important: the accessibility/ease-of-creation, or owning it?
It's also interesting because I haven't heard a ton of good legal arguments from people about why this shouldn't be the case, so there's a nontrivial chance that it holds in the future. I've heard tons of logic-based arguments: "it's creative because I decided what to generate, photographs are copyrightable, etc..." And those aren't necessarily bad arguments, they're just arguments that the US copyright office has already rejected. I haven't seen a lot of arguments about "here's why AI generated code would be eligible without a legal challenge overturning existing copyright rules."
----
[0]: https://www.theverge.com/2022/2/21/22944335/us-copyright-off...
[1]: https://aibusiness.com/ml/ai-generated-comic-book-loses-copy...
[2]: Not that the debate over the training data isn't interesting or important; that could also have implications for stuff like fair-use in transformative contexts. It's just that I think it's less likely to have far-reaching implications or be upheld.
It's also going to be hotly debated in the future, because right now most of the commercial AI-generators are just kind of ignoring this and at some point I think they're going to make the argument, "this law/interpretation can't hold or else it would be commercially devastating for us, and it needs to change."
But under current US copyright law, the US copyright office has pretty consistently ruled that:
- purely functional expressions without a creative aspect can't be copyrighted (ie, you patent an invention, you don't copyright it, and stuff like recipes aren't eligible for copyright at all, only the surrounding text is).
- AI-generated content does not have a creative aspect and isn't eligible for copyright protection.
They've even gone so far as to revoke copyright protections to AI-generated content[0][1]. We haven't gotten a similar ruling about code that I'm aware of, but given that the US copyright office already only grants copyright to code under the assumption that coding is a creative act, it's very difficult to imagine code having more legal protections than an image or a book.
----
In my opinion, this is the much more interesting debate than model's source[2], because nobody is really talking about it, and depending on how the inevitable legal challenges play out, it could completely change the field. It's an interesting choice to think about -- let's say an AI gets good enough that you can sit down, describe an app, and the AI just generates it straight up. Would you still use that tool even if the resulting code was public domain and you couldn't assert ownership over it?
Which would be more important: the accessibility/ease-of-creation, or owning it?
It's also interesting because I haven't heard a ton of good legal arguments from people about why this shouldn't be the case, so there's a nontrivial chance that it holds in the future. I've heard tons of logic-based arguments: "it's creative because I decided what to generate, photographs are copyrightable, etc..." And those aren't necessarily bad arguments, they're just arguments that the US copyright office has already rejected. I haven't seen a lot of arguments about "here's why AI generated code would be eligible without a legal challenge overturning existing copyright rules."
----
[0]: https://www.theverge.com/2022/2/21/22944335/us-copyright-off...
[1]: https://aibusiness.com/ml/ai-generated-comic-book-loses-copy...
[2]: Not that the debate over the training data isn't interesting or important; that could also have implications for stuff like fair-use in transformative contexts. It's just that I think it's less likely to have far-reaching implications or be upheld.
Someone demonstrated a Redux backend simulated by GPT-3. Others demonstrated front end design run by GPT-3. Maybe in the future we're going to ask, the model is going to whip up an app, we use it once and throw it away. One time use apps. They are getting comodified.
Nit to pick. Code cannot be owned. Rights to code may be owned. They are not the same. Otherwise, you’re spot on in my book.
Good point
> The output of a trained AI is insufficiently creative to be copyrighted. Only humans can hold a copyright.
That's overgeneralisation. A language model alone, yes, is just derivative. But a language model trained on solving problems with reinforcement learning can surpass humans. For example AlphaGo and AlphaTensor are models that learned from running simulations.
That's overgeneralisation. A language model alone, yes, is just derivative. But a language model trained on solving problems with reinforcement learning can surpass humans. For example AlphaGo and AlphaTensor are models that learned from running simulations.
It's not a question of whether the AI can be creative in a philosophical sense, current copyright law (in the US) only protects human creativity -- AI and non-human animals can't hold IP at all, nor can they delegate it to someone else.
Under current copyright law, if you built a model that surpassed humans and trained itself on simulations and it wasn't making derivative works but instead producing purely original output, the resulting output would not be copyrightable.
In fact, under current copyright law, if you built an AI that was sapient, the creative work it made would not be copyrightable -- unless you literally got the law to grant it legal personhood or something.
Moving AI in a direction of being more fundamentally creative makes this harder, not easier. The legal defense of copyright in cases involving AI is that the creativity is coming from the human using the AI, not the AI itself. As the AI gets better at generating usable output with less directing and micromanaging, that argument gets weaker, not stronger. The question isn't whether or not the AI is creative or produces novel output, the question is whether a human exhibited enough creativity making the work for it to meet the standard of gaining copyright protections -- which, traditionally, the US copyright office has usually held isn't true for AI-generated content.
Under current copyright law, if you built a model that surpassed humans and trained itself on simulations and it wasn't making derivative works but instead producing purely original output, the resulting output would not be copyrightable.
In fact, under current copyright law, if you built an AI that was sapient, the creative work it made would not be copyrightable -- unless you literally got the law to grant it legal personhood or something.
Moving AI in a direction of being more fundamentally creative makes this harder, not easier. The legal defense of copyright in cases involving AI is that the creativity is coming from the human using the AI, not the AI itself. As the AI gets better at generating usable output with less directing and micromanaging, that argument gets weaker, not stronger. The question isn't whether or not the AI is creative or produces novel output, the question is whether a human exhibited enough creativity making the work for it to meet the standard of gaining copyright protections -- which, traditionally, the US copyright office has usually held isn't true for AI-generated content.
I'd like to hear how you decide which piece of code is creative and which isn't.
It's for a court of law to decide. GP was merely stating (what sounds like a fairly accurate) description of how copyright on code currently works in most jurisdictions.
GP was stating that AI couldn't own copyright because it wasn't creative enough. Now you're telling me (and I agree) that whether it's creative enough is up to a court of law to decide. Feels like we're agreeing then, and disagreeing with GP?
AI can’t own copyright because only humans can get copyrights.
perhaps. but incentives can exist for people to publish their code as open source for the benefit of humanity. like credit and public recognition.
I would think copyright law would have something to say about what is or is not a derivative work. How similar to code snippets have to be (or claim to be) to be considered infringement on original source?
Outside of AI models, copy/pasting snippets from the likes of StackOverflow is already on unsteady ground. The threshold to bother with (and win) legal fights is pretty high. AI is catalyzing some kind of slow revolution in what "ownership" is, but there doesn't seem to be any definition that would _always_ satisfy common sense. Even if github yields and adds attribution or filters on license types, there's still a massive trove of knotted data in all these newer text/language/image/code models. Most of it is not truly public domain.. and a lot of it is wholly private and would involve contravention of regs like GDPR if used. Data used to feed these huge models is, sometimes (technically speaking...) stolen, commandeered, swiped from unassuming creators. Many artists the world over are rightfully furious about DALL-E and Stable Diffusion. I wonder where this will all end!
AI needs to be regulated somehow. Precedent needs to be set.
AI needs to be regulated somehow. Precedent needs to be set.
>Many artists the world over are rightfully furious about DALL-E and Stable Diffusion
I struggle with this one. How are these models different from the typical human creative process of:
1. look at lots of existing art to get inspiration
2. select components from several different styles & add your own flair
3. call the output an "original" painting in your own style
Of course I see the other side as well. These models are just a large multi-dimensional function of all of their input data, which means they ought to be derivative works of all of their inputs, and their outputs should thus also be derivative works of the inputs.
It feels unsatisfying (and a bit naive) to say that filtering a bunch of data through a neural network somehow clears the copyright of that original data, but on the other hand, the AI revolution depends on that interpretation.
I struggle with this one. How are these models different from the typical human creative process of:
1. look at lots of existing art to get inspiration
2. select components from several different styles & add your own flair
3. call the output an "original" painting in your own style
Of course I see the other side as well. These models are just a large multi-dimensional function of all of their input data, which means they ought to be derivative works of all of their inputs, and their outputs should thus also be derivative works of the inputs.
It feels unsatisfying (and a bit naive) to say that filtering a bunch of data through a neural network somehow clears the copyright of that original data, but on the other hand, the AI revolution depends on that interpretation.
In a word scale. Scale matters. How is one locust different to a million? How is hand copying a manuscript (highly controlled in medieval Europe) different to a printing press. These models will and are already having a profound economic impact on the creative sectors they mimic. Whether existing law applies is a specialist question. But they're not remotely comparable from a pragmatic perspective with pieces manually created by individuals taking 'inspiration' from existing work. A closer parallel would be the industrialisation of painting duplication which is done in Chinese 'artist villages' like Dafen. It's a strange kind of equanimity to shrug while others lose their livelihood and have their self expression systematically duplicated and commodified. No kidding they're furious. That you can't empathise or find it 'naive' is your issue. What attitude would be more understandable? Resignation? Calmly retraining to work in a call centre?
>A closer parallel would be the industrialisation of painting duplication
But they directly reproduce the source material.
AI art they clearly does not.
The luddites seem a better parallel when it comes to scale. Where a machine comes along capable of producing in much higher quantities and in much greater efficiencies.
Or perhaps photography? Also a fear of scale. For a long time photographers were not considered artists. And there were calls to tamp down on this innovation for the harm it would cause.
I don't suppose anyone back then could imagine now how photography is used. Perhaps they would feel it much better that a tiny artist quickly sketch out the thing my phone was looking at.
But they directly reproduce the source material.
AI art they clearly does not.
The luddites seem a better parallel when it comes to scale. Where a machine comes along capable of producing in much higher quantities and in much greater efficiencies.
Or perhaps photography? Also a fear of scale. For a long time photographers were not considered artists. And there were calls to tamp down on this innovation for the harm it would cause.
I don't suppose anyone back then could imagine now how photography is used. Perhaps they would feel it much better that a tiny artist quickly sketch out the thing my phone was looking at.
photography is actually a very good example of how this talking point breaks down.
there are a great deal of laws and social codes around where you can and cannot film and take photographs. photography is explicitly disallowed in most museums and art galleries. filming in public requires permits and waivers in many places, especially if the filming is done with a commercial interest.
'but how is capturing an image with a camera lens any different from capturing it with my retina? both are simply methods of refracting light and translating that information into a storage medium. the camera is just remembering* like a human remembers and we mustn't let luddites regulate this revolutionary technology or we'll impede the progress of the camera becoming an even more better rememberer'*
the social ramifications of the camera are still being reckoned with to this day and yes, cameras very highly regulated compared to pretty much any other human method of expression or transcription, in fact we are still coming up with new things you cannot film eg the recent crop of revenge porn laws.
'but how is capturing an image with a camera lens any different from capturing it with my retina? both are simply methods of refracting light and translating that information into a storage medium. the camera is just remembering* like a human remembers and we mustn't let luddites regulate this revolutionary technology or we'll impede the progress of the camera becoming an even more better rememberer'*
the social ramifications of the camera are still being reckoned with to this day and yes, cameras very highly regulated compared to pretty much any other human method of expression or transcription, in fact we are still coming up with new things you cannot film eg the recent crop of revenge porn laws.
I had the same thoughts.
Finally, someone has put into words the logical end of the trite photography analogy.
It's a refreshingly sane perspective to see in this thread.
Finally, someone has put into words the logical end of the trite photography analogy.
It's a refreshingly sane perspective to see in this thread.
AI art they clearly does not.
It does not 1-1 recreate source material, but if we didn't have human artists anymore and replaced them all with the current stock of AI we would have no more developing art movements. It is wholly uncreative in a way that humans are not, and it does not understand what humans would find interesting, only what humans have already made. This makes it a pretty good tool for some amateurish applications or places where quality isn't important, but it is not a 1-1 copy of a human capable of making human art.
It does not 1-1 recreate source material, but if we didn't have human artists anymore and replaced them all with the current stock of AI we would have no more developing art movements. It is wholly uncreative in a way that humans are not, and it does not understand what humans would find interesting, only what humans have already made. This makes it a pretty good tool for some amateurish applications or places where quality isn't important, but it is not a 1-1 copy of a human capable of making human art.
>it is wholly uncreative in a way that humans are not
The work I've seen manipulating, weilding, and modifying this new tool, has been very creative In painting, reprompting, texture and uv map generation, dreambooth and dedicated fine tuning. Integrating these workflows into human directed procedures to produce in hours what would take days, sometimes months, sometimes not even within the scope of human capability.
If you think the only thing here is people jibbering at a black box and posting 512*512 images, I can see why dumping the entire thing would seem reasonable.
The work I've seen manipulating, weilding, and modifying this new tool, has been very creative In painting, reprompting, texture and uv map generation, dreambooth and dedicated fine tuning. Integrating these workflows into human directed procedures to produce in hours what would take days, sometimes months, sometimes not even within the scope of human capability.
If you think the only thing here is people jibbering at a black box and posting 512*512 images, I can see why dumping the entire thing would seem reasonable.
>AI art they clearly does not.
https://twitter.com/kortizart/status/1588915427018559490
I would think these are close enough that any human that produced that output could be claimed to have plagiarized.
https://twitter.com/kortizart/status/1588915427018559490
I would think these are close enough that any human that produced that output could be claimed to have plagiarized.
ugh, how tedious.
Famous images prevelant in the dataset that are trained against many times and the directly requested are produced as is, because that's litterally what was requested and the model was trained against them.
There's only one way to draw The Mona Lisa. These AI's are not used for this purpose, nor is any of this core to their functioning. And I have to assume you know that.
Famous images prevelant in the dataset that are trained against many times and the directly requested are produced as is, because that's litterally what was requested and the model was trained against them.
There's only one way to draw The Mona Lisa. These AI's are not used for this purpose, nor is any of this core to their functioning. And I have to assume you know that.
> In a word scale. Scale matters
> to shrug while others lose their livelihood and have their self expression systematically duplicated and commodified.
The same exact arguments were made when the camera was invented.
In order to be consistent, you would have to argue that the camera should have been banned to protect painters.
> That you can't empathise or find it 'naive'
Oh we understand. We just know that it is the exact same argument, that ludites make every single time a new technology is invented, that increases efficiency.
It is always the same argument repeated over and over again. And if those arguments weren't good in the past, we aren't going to listen to defeated arguments now.
> to shrug while others lose their livelihood and have their self expression systematically duplicated and commodified.
The same exact arguments were made when the camera was invented.
In order to be consistent, you would have to argue that the camera should have been banned to protect painters.
> That you can't empathise or find it 'naive'
Oh we understand. We just know that it is the exact same argument, that ludites make every single time a new technology is invented, that increases efficiency.
It is always the same argument repeated over and over again. And if those arguments weren't good in the past, we aren't going to listen to defeated arguments now.
> filtering a bunch of data through a neural network somehow clears the copyright of that original data
There are two kinds of data - the idea and the expression. You can protect expression, but can't stop AI from learning the ideas. This is not "clearing the copyright", it is fair use of the data. Eventually it will even learn how many fingers and heads to draw on a human - the kind of "idea data" you can't copyright.
We should stop mixing together idea and expression. Ideas are not owned by anyone. The real question is what criteria and threshold to use for judging infringement, so AI people can get on to making safe models.
Besides ideas, you can't copyright recipes, APIs and purely functional/obvious code.
There are two kinds of data - the idea and the expression. You can protect expression, but can't stop AI from learning the ideas. This is not "clearing the copyright", it is fair use of the data. Eventually it will even learn how many fingers and heads to draw on a human - the kind of "idea data" you can't copyright.
We should stop mixing together idea and expression. Ideas are not owned by anyone. The real question is what criteria and threshold to use for judging infringement, so AI people can get on to making safe models.
Besides ideas, you can't copyright recipes, APIs and purely functional/obvious code.
> stolen, commandeered, swiped from unassuming creators
swiped from unassuming creators who put it online for everyone and Google Bot to see? I presume they follow robots.txt when they crawl
swiped from unassuming creators who put it online for everyone and Google Bot to see? I presume they follow robots.txt when they crawl
I don’t believe stack overflow is in the same grey area, they have a pretty clear TOS and license for the code snippets. I’m pretty sure it’s CC-BY-SA.
Regardless of what side you stand on, this case is sorely needed, so everybody knows what the legalities are.
[deleted]
This article adds no new information.
We knew it is hotly debated, and we have heard both sides.
Just possibly there are people out there who haven't heard both sides and perhaps they are the intended audience for this?
And maybe some of them even read HN occasionally and are currently enjoying the article.
And maybe some of them even read HN occasionally and are currently enjoying the article.
ownership is a concept in dire need of revision
Once we have AI-generated laws I'm sure this will be sorted out.
Yes, as long as it doesn't disregard the original creator's intentions in the formulation.
Even that notion is tricky in the context of text AI. It doesn't seem obviously clear to me what the implied stance towards AI training would be for many popular license types. It seems more like the creators of many licenses didn't explicitly consider this use case.
And the more meta question is whether the creators' rights should even extend to that realm. Can you specify in a license "This text must not be read by students in the context of an educational course" (which is arguably the closest analogue to AI training)? If that's not possible, then where do we draw the line between tools-assisted learning by humans and human tools learning on their own? You can arrive at answers, but it seems to me like there are many possible reasonable interpretations.
And the more meta question is whether the creators' rights should even extend to that realm. Can you specify in a license "This text must not be read by students in the context of an educational course" (which is arguably the closest analogue to AI training)? If that's not possible, then where do we draw the line between tools-assisted learning by humans and human tools learning on their own? You can arrive at answers, but it seems to me like there are many possible reasonable interpretations.
Why is it fair use if you train a model on copyrighted material and use its transformative output, despite going against the will of the author?
Can we all legally pirate educational books since it's for self training and producing transformative outputs? Can I consume all media (books, movies, music) the same way, and call it fair use? Also for software?
Can we all legally pirate educational books since it's for self training and producing transformative outputs? Can I consume all media (books, movies, music) the same way, and call it fair use? Also for software?
Content that hasn't been legally obtained wouldn't be legal to consume in any way. That's not what I meant, and I think that's a bit of a disingenuous interpretation of my comment.
The debate is about content that is generally legally obtained, but might come with certain restrictions. Where restrictions is a broad term and might also just come in the form of a copyleft license, eg. My main point was that in many situations, eg involving open source licenses, it's really not clear from the terms what the creator's intent regarding AI training was. And the broader question is whether training is fair use, or something else, maybe even a new legal concept that would have to be established. Or, what's the difference between art students going to the museum to be inspired, and Dall-E 'looking' at public domain images?
The debate is about content that is generally legally obtained, but might come with certain restrictions. Where restrictions is a broad term and might also just come in the form of a copyleft license, eg. My main point was that in many situations, eg involving open source licenses, it's really not clear from the terms what the creator's intent regarding AI training was. And the broader question is whether training is fair use, or something else, maybe even a new legal concept that would have to be established. Or, what's the difference between art students going to the museum to be inspired, and Dall-E 'looking' at public domain images?
I acknowledge I stretched a bit your comment. I meant it as a way to try to find the line on what's OK with these new AIs rather than ill-intended O:)
Regarding your point, I wonder how different is it to break the "terms of use" for fair right, vs breaking the "terms of obtaining" alltogether. They both are about jumping over owner's will, who has the full rights of the work.
I very much look forward to see how the ethics and law around these issues evolve.
Regarding your point, I wonder how different is it to break the "terms of use" for fair right, vs breaking the "terms of obtaining" alltogether. They both are about jumping over owner's will, who has the full rights of the work.
I very much look forward to see how the ethics and law around these issues evolve.
yeah, I struggle to see how training AI on illegally gotten data is legal.
> (which is arguably the closest analogue to AI training)
It would be more accurate to compare those students with the corporate interests involved in AI training, not the model's use of the content.
It would be more accurate to compare those students with the corporate interests involved in AI training, not the model's use of the content.
The original creator's intentions do not override IP law and neither should they.
Otherwise any deluded fool with extreme views on copyright could claim ownership way beyond that which the law currently offers.
Otherwise any deluded fool with extreme views on copyright could claim ownership way beyond that which the law currently offers.
Sure, respecting intellectual property laws is exactly what must be considered here.
Agreed.
You don't mind if I borrow your car do you?
And your house?
Let's not quibble about the particulars of me ever giving them back.
You don't mind if I borrow your car do you?
And your house?
Let's not quibble about the particulars of me ever giving them back.
I don't want you taking my car because I need to use my car and only one of us can use it at a time. Intellectual property can be endlessly copied without anyone losing anything. This is not a defense of IP theft (and it's pretty reasonable to think that IP law serves an important purpose protecting creatives' ability to get compensated for their work) but it means that they require a fundamentally different approach.
> ownership is a concept in dire need of revision
GP did not limit the scope of their opinion on ownership to IP.
GP did not limit the scope of their opinion on ownership to IP.
indeed, the first step (IMO) towards revising the notion is to recognize that physical (material, tangible) assets inherently work differently than digital assets.
As I understand so far the main reason to seek a revision of the concept of ownership is exactly due to the existence (enabled by internet technology) of digital assets.
copy-pasting is HOW computers work. copy-pasting does not do well in society ruled by the exclusivity-mindset inherent to marketplaces (and their societies) of tangible assets
As I understand so far the main reason to seek a revision of the concept of ownership is exactly due to the existence (enabled by internet technology) of digital assets.
copy-pasting is HOW computers work. copy-pasting does not do well in society ruled by the exclusivity-mindset inherent to marketplaces (and their societies) of tangible assets
I'm not exactly an IP hawk, from from it.
I agree with you that digital "asset" ownership is new, unexplored territory for humans.
We've never been able to separate the content from the distribution medium before now, and we're struggling to recreate a model we're familiar with (physical media) by imposing absurdities like DRM.
NFTs are also an absurd way of trying to solve the same problem.
The issue we wrestle with is mistaking "ownership" with a access.
I can own a physical book, but I do not own the content of the book, I've paid for access to that content and the book is the medium.
Digital assets are the same, I don't own them, they are a form of access to content that someone else owns, and has granted me an either implicit or explicit license to use.
I think this is the concept of ownership that needs to be revised.
Rather than a mp3 being treated, and thought of, like a book - it should be thought of more as a movie ticket. Something that grants me access to content within the limits defined by the content owner.
I agree with you that digital "asset" ownership is new, unexplored territory for humans.
We've never been able to separate the content from the distribution medium before now, and we're struggling to recreate a model we're familiar with (physical media) by imposing absurdities like DRM.
NFTs are also an absurd way of trying to solve the same problem.
The issue we wrestle with is mistaking "ownership" with a access.
I can own a physical book, but I do not own the content of the book, I've paid for access to that content and the book is the medium.
Digital assets are the same, I don't own them, they are a form of access to content that someone else owns, and has granted me an either implicit or explicit license to use.
I think this is the concept of ownership that needs to be revised.
Rather than a mp3 being treated, and thought of, like a book - it should be thought of more as a movie ticket. Something that grants me access to content within the limits defined by the content owner.
but this essentially restricts the spread of the 'digital boon' (referring the the new possibilities afforded by this new technologies)
I'm trying to point out that this distinction (ownership as distinct from access) leads towards a capture of the digital advantage by people with better leverage.
moreover, I disagree that if I own a book I do not own the contents of it. the mindset that I don't seems too close to saying that I can know things (well understood learned concepts) but still somehow not own them.
This in my view is like a 'hook' which pulls towards the reality that somebody else owns the contents of my own mind, hence that I do not own that part of myself. I hope you see where I'm going with this and why I find your posture troubling.
It's only a few short (conceptual) steps from doing away with individual freedom for the sake of what?
I'm trying to point out that this distinction (ownership as distinct from access) leads towards a capture of the digital advantage by people with better leverage.
moreover, I disagree that if I own a book I do not own the contents of it. the mindset that I don't seems too close to saying that I can know things (well understood learned concepts) but still somehow not own them.
This in my view is like a 'hook' which pulls towards the reality that somebody else owns the contents of my own mind, hence that I do not own that part of myself. I hope you see where I'm going with this and why I find your posture troubling.
It's only a few short (conceptual) steps from doing away with individual freedom for the sake of what?
> You don't mind if I borrow your car do you?
ZipCar, et. al.
> And your house?
AirBnB
> Let's not quibble about the particulars of me ever giving them back.
Symmetry.
ZipCar, et. al.
> And your house?
AirBnB
> Let's not quibble about the particulars of me ever giving them back.
Symmetry.
Historically rent-seeking has been a negative for the health of society.
Granting overlords the power of ownership, reducing individuals to renters that must comply with the whims of the property masters, is not a path to increased liberty.
Granting overlords the power of ownership, reducing individuals to renters that must comply with the whims of the property masters, is not a path to increased liberty.
> Historically rent-seeking has been a negative for the health of society.
Sure. I mean, I'm not an economist nor a sociologist, I bet there's some nuance to that, eh?
> Granting overlords the power of ownership, reducing individuals to renters that must comply with the whims of the property masters
Peasants and serfs. In re: technology I talk about Morlocks and Eloi. Yeah I think that's a lousy way to structure society.
The original point that you replied to was, "ownership is a concept in dire need of revision" and it sounds to me like that's where you're coming from too?
Sure. I mean, I'm not an economist nor a sociologist, I bet there's some nuance to that, eh?
> Granting overlords the power of ownership, reducing individuals to renters that must comply with the whims of the property masters
Peasants and serfs. In re: technology I talk about Morlocks and Eloi. Yeah I think that's a lousy way to structure society.
The original point that you replied to was, "ownership is a concept in dire need of revision" and it sounds to me like that's where you're coming from too?
Yes, like scientific grounding or abandonment.
If only we held all myths to such high esteem as we societally hold ownership.
If only we held all myths to such high esteem as we societally hold ownership.
The only way to truly own something, is to either share it or destroy it.
I really like this, it rings so true to me in the current times.
Temporal everything, air, food; a life even-keel.
or abolition
but to do that without any reasonable social support nets is absurd
but to do that without any reasonable social support nets is absurd
but what does it mean to destroy the concept of ownership (i.e. the notion of property)
if we take that
> The only way to truly own something, is to either share it or destroy it.
could I argue that then, we would be taking ownership of the concept of ownership??
ahahha... I think this is kind of funny. And I'd admit that it's not very helpful to the goal of revising the very concept of ownership.
my own parent comment is already controversial with many replies and exactly 0 points (at the time of this response)
if we take that
> The only way to truly own something, is to either share it or destroy it.
could I argue that then, we would be taking ownership of the concept of ownership??
ahahha... I think this is kind of funny. And I'd admit that it's not very helpful to the goal of revising the very concept of ownership.
my own parent comment is already controversial with many replies and exactly 0 points (at the time of this response)
private property successfully didn't exist in various societies, including larger scale societies and ones that had discovered agriculture, and not only under communist regimes. it does mean to share, without ownership.
riedel(2)
Ideas cannot be owned. People can be owned, if you have slavery.
Everyone must unlearn the term "Intellectual Property". These laws are anti-property rights. They are Intellectual Slavery laws (https://breckyunits.com/an-unpopular-phrase.html).
The United States government employs more knowledge workers than all other companies (see NIH, DoD, CDC, NASA, NOAA, NWS, et cetera). Everything they produce is public domain, by law. And yet, the people producing these information products still get paid!
We don't need (c)opywrong laws. We don't need Intellectual Slavery laws. We still have cotton even after the 13th Amendment (we actually have more and better cotton now), and we will still have creative works after the passing of the Intellectual Freedom Amendment (we actually will have more and better creative works) - https://breckyunits.com/the-intellectual-freedom-amendment.h....
Everyone must unlearn the term "Intellectual Property". These laws are anti-property rights. They are Intellectual Slavery laws (https://breckyunits.com/an-unpopular-phrase.html).
The United States government employs more knowledge workers than all other companies (see NIH, DoD, CDC, NASA, NOAA, NWS, et cetera). Everything they produce is public domain, by law. And yet, the people producing these information products still get paid!
We don't need (c)opywrong laws. We don't need Intellectual Slavery laws. We still have cotton even after the 13th Amendment (we actually have more and better cotton now), and we will still have creative works after the passing of the Intellectual Freedom Amendment (we actually will have more and better creative works) - https://breckyunits.com/the-intellectual-freedom-amendment.h....
People have a strange tendency to enjoy being rewarded for work they've done.
I suspect that the folks who invest hundreds of millions of dollars into production costs for a movie, rather enjoy the ability to recoup those costs by restricting access to only those who are willing to pay for the privilege.
I suspect that the folks who invest hundreds of millions of dollars into production costs for a movie, rather enjoy the ability to recoup those costs by restricting access to only those who are willing to pay for the privilege.
> People have a strange tendency to enjoy being rewarded for work they've done.
Did you not read: "The United States government employs more knowledge workers than all other companies (see NIH, DoD, CDC, NASA, NOAA, NWS, et cetera). Everything they produce is public domain, by law. And yet, the people producing these information products still get paid!"
These laws are counterproductive and have terrible second order effects, and are cruel to everyone except a tiny % of the population, just like slavery laws.
Did you not read: "The United States government employs more knowledge workers than all other companies (see NIH, DoD, CDC, NASA, NOAA, NWS, et cetera). Everything they produce is public domain, by law. And yet, the people producing these information products still get paid!"
These laws are counterproductive and have terrible second order effects, and are cruel to everyone except a tiny % of the population, just like slavery laws.
but what about people getting paid for the invested money of movies made by their ancestors, dead over 50 years ago (and up to more than 100 years ago, whatever is the current copyright term for movies)
do they have a right to enjoy being rewarded for work they didn't do but their ancestors did?
if the intention of those laws was to encourage creativity, how come they're stiffing it more than encouraging without any signs of any government trying to correct the laws to better match their purported intentions?
do they have a right to enjoy being rewarded for work they didn't do but their ancestors did?
if the intention of those laws was to encourage creativity, how come they're stiffing it more than encouraging without any signs of any government trying to correct the laws to better match their purported intentions?
If your argument is about the absurdity of current copyright terms, you'll find no disagreement here.
We should have stuck with the original 14 + 14 as laid out in the constitution.
We should have stuck with the original 14 + 14 as laid out in the constitution.
Is it really feasible? What does “traces attribution” even mean here? It’s not emitting “code”, it’s emitting individual tokens that each were found throughout the input corpus. The “code” is the arrangement of those tokens, but that is determined by the weighting of the whole network, so what can be traced?
Can someone who understands generative ML better than me weigh in on this?