The music industry's A.I. experiments(newyorker.com)
newyorker.com
The music industry's A.I. experiments
https://www.newyorker.com/magazine/2024/02/05/inside-the-music-industrys-high-stakes-ai-experiments
60 comments
https://archive.ph/Qeide
The article seems to focus on A.I in relation to big artists and pop music (and that one rich CEO) but the first casualties will be the "stock" music market, and then TV shows & movies composers / producers / artists.
It's not totally there yet but it's improving.
Is it already happening in the stock photos world?
> the music industry (....) now appears to be more profitable than ever.
Tell that to the music professionals actually producing that music.
It's not totally there yet but it's improving.
Is it already happening in the stock photos world?
> the music industry (....) now appears to be more profitable than ever.
Tell that to the music professionals actually producing that music.
I played music from a young age, studied Jazz and honestly, the hope of making a decent career out of music died about 50 years ago with the advent of recordings and high quality audio equipment, once live musicians were no longer required to have music in commercial settings, it was over because the average punter, who knows really nothing about art, couldn't give a shit what they're listening too. The side effect here is that with this change, the incentive for brilliant musicians to exist died too, and so we're left with the seriously diminished music we listen too today.
When is the last time you heard a Stan Getz equivalent?
Maybe there is still some more raping to be done by "the industry", but the whole scene has been raped and pillaged quite devastatingly already.
There honestly seems to be a strong and enduring resurgence in live music popularity, which IMO is exciting, and I think will continue as the quality of commercial music just seems to go down the toilet year on year.
The other issue generative AI will have is the same problem most musicians have, people like familiarity, you can making incredible music and 99% of ears will have no interest in it because it's unrelated to anything they, or they're fiends have listened too while going on that summer road trip in college. Why do people still listen to Vanilla Ice?
Music is a wonderful, creative, artistic pursuit, just don't expect to make too much money out of it, and you'll be in line with reality. I play guitar for my son when he can't sleep, and it's the most rewarding thing I've ever done musically.
When is the last time you heard a Stan Getz equivalent?
Maybe there is still some more raping to be done by "the industry", but the whole scene has been raped and pillaged quite devastatingly already.
There honestly seems to be a strong and enduring resurgence in live music popularity, which IMO is exciting, and I think will continue as the quality of commercial music just seems to go down the toilet year on year.
The other issue generative AI will have is the same problem most musicians have, people like familiarity, you can making incredible music and 99% of ears will have no interest in it because it's unrelated to anything they, or they're fiends have listened too while going on that summer road trip in college. Why do people still listen to Vanilla Ice?
Music is a wonderful, creative, artistic pursuit, just don't expect to make too much money out of it, and you'll be in line with reality. I play guitar for my son when he can't sleep, and it's the most rewarding thing I've ever done musically.
You're right that it's not there yet, but it's scarily close.
I used a generative AI music tool to create ~1 minute snippets of fully mixed music. The tool offers 2 fields: Lyrics and a style prompt. From the initial audio you can generate continuations from them, creating different branches and there's some syntax to specify structure too ([chorus], [verse], [outro], (instrumental) etc). The quality of the compositions is quite good, though the choices it makes can be rather random just like any generative AI but that isn't always unwelcome.
From there I picked the best and most interesting bits of audio, split them apart those apart into separate tracks (vocals, guitar, percussion) with another AI. At that point I can bring them into a music sequencer, pull them apart, time stretch, rearrange, anything. You can generate an instrumental with a lead guitar, pull it out and sprinkle it through the entire song. Re-spin vocal deliveries, mix and match them.
This is a draft of something I've been working on: https://voca.ro/15yurCvoA9yR
The main drawback is the quality, it's like a mediocre cassette tape recording much of the time and splitting it up into tracks causes some generation loss.
I used a generative AI music tool to create ~1 minute snippets of fully mixed music. The tool offers 2 fields: Lyrics and a style prompt. From the initial audio you can generate continuations from them, creating different branches and there's some syntax to specify structure too ([chorus], [verse], [outro], (instrumental) etc). The quality of the compositions is quite good, though the choices it makes can be rather random just like any generative AI but that isn't always unwelcome.
From there I picked the best and most interesting bits of audio, split them apart those apart into separate tracks (vocals, guitar, percussion) with another AI. At that point I can bring them into a music sequencer, pull them apart, time stretch, rearrange, anything. You can generate an instrumental with a lead guitar, pull it out and sprinkle it through the entire song. Re-spin vocal deliveries, mix and match them.
This is a draft of something I've been working on: https://voca.ro/15yurCvoA9yR
The main drawback is the quality, it's like a mediocre cassette tape recording much of the time and splitting it up into tracks causes some generation loss.
What's the name of the tool where you can specify the structure of the song?
suno.ai is the generator I used, structure is mostly implied through the lyrics rather than directly controlled though. But given that you can break down the generated audio into it's constituent elements with something like moises.ai or RipX DAW, you can rebuild it into any structure you want.
> Is it already happening in the stock photos world?
I think for stock photos it's still early days. But soon when AI technology becomes so widespread that it's accessible to everyone, people will stop using stock photos. Especially if the level of image generation reaches almost perfection. After all with AI you can create anything without spending time searching for stock photos online
>Tell that to the music professionals actually producing that music.
I mean, that's how capitalism, and in particular creative industry, has always worked. Value is captured by management, not the people who create it. The celebrities who can afford to negotiate million dollar contracts and retain rights to their work are the exception, not the rule.
I mean, that's how capitalism, and in particular creative industry, has always worked. Value is captured by management, not the people who create it. The celebrities who can afford to negotiate million dollar contracts and retain rights to their work are the exception, not the rule.
Look for desperate, heavily funded lobbying efforts to create new expansions of copyright, or something like it.
LLM-based music generators do what musicians do - listen to a lot of stuff and then generate their own music. Most popular music isn't that original. It's mostly about branding.
LLM-based music generators do what musicians do - listen to a lot of stuff and then generate their own music. Most popular music isn't that original. It's mostly about branding.
> LLM-based music generators do what musicians do - listen to a lot of stuff and then generate their own music.
I love this analogy, because it looks so benign, while being so off.
Question: How many hours of music an LLM can listen and ingest in an hour?
Answer: Probably 10x more than a human, and will ingest it way better in one go than a human who needs 5-10x concentrated listens to completely understand what's going on at all layers.
So, in practice, an LLM can ingest 50-100x more music in unit time, when compared to a human.
"An LLM is like a musician. Just listens". Indeed, that's true, if I wince really hard, so my head hurts.
BTW, I'm all for respecting copyright, and no, AI training is not fair use, because AI is not a human who consumes reasonable amounts of information and understands and derives it in a unit time.
I love this analogy, because it looks so benign, while being so off.
Question: How many hours of music an LLM can listen and ingest in an hour?
Answer: Probably 10x more than a human, and will ingest it way better in one go than a human who needs 5-10x concentrated listens to completely understand what's going on at all layers.
So, in practice, an LLM can ingest 50-100x more music in unit time, when compared to a human.
"An LLM is like a musician. Just listens". Indeed, that's true, if I wince really hard, so my head hurts.
BTW, I'm all for respecting copyright, and no, AI training is not fair use, because AI is not a human who consumes reasonable amounts of information and understands and derives it in a unit time.
> AI training is not fair use, because AI is not a human who consumes reasonable amounts of information and understands and derives it in a unit time.
Please illustrate why "LLMs ingest training data faster than humans and have better recall" has anything to do with fair use.
Please illustrate why "LLMs ingest training data faster than humans and have better recall" has anything to do with fair use.
From [0],
Notwithstanding the provisions of sections 17 U.S.C. § 106 and 17 U.S.C. § 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include:
---
Let's polish over the "scholarship, or research" clause for a moment, because none of the commercial entities are doing it for research. Assuming the fair use clause didn't fail there, let's focus on two important points:
Whatever you do, you can create derivative works of the ingested works, or derivative works containing ingested parts verbatim at great speed with this "Shiny LLM thing".
A human neither can ingest that amount of information, nor retain all of it that perfectly, hence limiting the inspiration from these works, and forces one to add their own original ideas or inspirations rooted in other sensory inputs (feelings, sight, own experiences, etc.), not reducing the value of the work used, or not damaging the creator of the work the creator got inspiration.
Incidentally, we have a word for cases this inspiration gets a bit too far: "plagiarism".
From that point, an LLM, which remembers everything verbatim and mixing them together is arguably making plagiarism from many sources to create their own works, while being unable to cite them accurately. A very useful property, indeed.
*: I know how an LLM stores its data in its weights, but even without the data itself, the weights can construct the training data verbatim in many cases, so it stores its training data. Not in its original form, but in a derivative and reversible form, not unlike compression.
[0]: https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors
Notwithstanding the provisions of sections 17 U.S.C. § 106 and 17 U.S.C. § 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include:
1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
2. the nature of the copyrighted work;
3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
4. the effect of the use upon the potential market for or value of the copyrighted work.
---
Let's polish over the "scholarship, or research" clause for a moment, because none of the commercial entities are doing it for research. Assuming the fair use clause didn't fail there, let's focus on two important points:
3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
4. the effect of the use upon the potential market for or value of the copyrighted work.
Whatever you do, you can create derivative works of the ingested works, or derivative works containing ingested parts verbatim at great speed with this "Shiny LLM thing".
A human neither can ingest that amount of information, nor retain all of it that perfectly, hence limiting the inspiration from these works, and forces one to add their own original ideas or inspirations rooted in other sensory inputs (feelings, sight, own experiences, etc.), not reducing the value of the work used, or not damaging the creator of the work the creator got inspiration.
Incidentally, we have a word for cases this inspiration gets a bit too far: "plagiarism".
From that point, an LLM, which remembers everything verbatim and mixing them together is arguably making plagiarism from many sources to create their own works, while being unable to cite them accurately. A very useful property, indeed.
*: I know how an LLM stores its data in its weights, but even without the data itself, the weights can construct the training data verbatim in many cases, so it stores its training data. Not in its original form, but in a derivative and reversible form, not unlike compression.
[0]: https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors
> From that point, an LLM, which remembers everything verbatim and mixing them together is arguably making plagiarism from many sources to create their own works, while being unable to cite them accurately. A very useful property, indeed.
I think this is a weak point in your argument. I support the rest of your reasoning, but this point can be attacked. For plagiarism to exist, the original work has to be easily detectable by a human. We've seen this sometimes be the case (see NYT lawsuit), but often times, the line is blurry (see Ed Sheeran lawsuit [0]). There's a reasonable argument to make that the combination of existing ideas by an AI creates something novel. It's what arguably what artists do, too. There's even a famous book called "Steal like an artist" [1]. As such, some output by AI can be plagiarism, but the very way modern AI works may not automatically mean everything is plagiarism.
[0]: https://www.youtube.com/watch?v=NcCKlsTgjeM
[1]: https://austinkleon.com/steal/
I think this is a weak point in your argument. I support the rest of your reasoning, but this point can be attacked. For plagiarism to exist, the original work has to be easily detectable by a human. We've seen this sometimes be the case (see NYT lawsuit), but often times, the line is blurry (see Ed Sheeran lawsuit [0]). There's a reasonable argument to make that the combination of existing ideas by an AI creates something novel. It's what arguably what artists do, too. There's even a famous book called "Steal like an artist" [1]. As such, some output by AI can be plagiarism, but the very way modern AI works may not automatically mean everything is plagiarism.
[0]: https://www.youtube.com/watch?v=NcCKlsTgjeM
[1]: https://austinkleon.com/steal/
Someone needs to create a "Godwin's Law" for AI such that for any discussion of LLMs on the web, the chance that someone will equate LLMs to humans equals one.
No "expansion of copyright" is needed in music, the requirements in the US are already onerous enough for the entirely original human-written and performed Blurred Lines to be deemed a copyright violation for consciously emulating the "feel" of a Marvin Gaye baseline.
The people that need to change this status quo - like it or loathe it - are the people that want to profit from spamming streaming services with zero-effort derivatives of popular music (people that do it for fun or use 'AI' tools for post production or loop-variation will continue to be fine). I have a lot less sympathy with them than the many artists who've lost court cases for 'inspiration', and if they're relying on the whole "models learn just like people" canard they're going to be in a weaker position than usual, because not only is that deemed insufficient for people, but it's unambiguously untrue that machines "listen" like humans do (they're bulk processing files, not experiencing consonance or dissonance in the ear) or that they "perform" as opposed to stochastically transferring elements of style. Even the most cliched blues going has human reasons for note choice and intonation other than "weighted average of other blues with overlapping description"
The people that need to change this status quo - like it or loathe it - are the people that want to profit from spamming streaming services with zero-effort derivatives of popular music (people that do it for fun or use 'AI' tools for post production or loop-variation will continue to be fine). I have a lot less sympathy with them than the many artists who've lost court cases for 'inspiration', and if they're relying on the whole "models learn just like people" canard they're going to be in a weaker position than usual, because not only is that deemed insufficient for people, but it's unambiguously untrue that machines "listen" like humans do (they're bulk processing files, not experiencing consonance or dissonance in the ear) or that they "perform" as opposed to stochastically transferring elements of style. Even the most cliched blues going has human reasons for note choice and intonation other than "weighted average of other blues with overlapping description"
> Even the most cliched blues going has human reasons for note choice and intonation other than "weighted average of other blues with overlapping description"
From what I've heard about human cognition, we come up with a word-based justification after we decide what to do, it just that the research involved a brain scanner to find this out because it doesn't (usually[0]) feel like this is what's going on on the inside.
So the real reason may well be "weighted average of other blues with overlapping description", followed (not preceded!) by a language model producing some convincing words that can communicate the mental state to other humans.
IMO, the main difference between AI and humans right now, is that humans can do human things with far fewer examples. AI has the speed advantage that lets them read all the words in the world, listen to all the music ever recorder, watch all the videos ever filmed… but are so stupid[1] that they need to, too.
[0] I do sometimes notice this in my own brain: I can have a fully-formed sentence before my inner dialogue tries to (for a lack of a better word for the internal voice) "speak" it — if I then try to skip the inner voice because I've already had the thought and turning into an imagined voice is clearly a waste of time, the part of me which is the inner voice is, for lack of a better word, frustrated.
[1] Stupid on this particular axis. Intelligence isn't fantastically well defined even in humans, is badly defined cross-species, and doesn't have a singular definition at all for machines, so it is perfectly valid to consider intelligence to be a question of "what can it do?" rather than "how fast does it learn?", for which many current AI (there's no G, so narrow AI that only plays chess is fine) "smarter than humans".
From what I've heard about human cognition, we come up with a word-based justification after we decide what to do, it just that the research involved a brain scanner to find this out because it doesn't (usually[0]) feel like this is what's going on on the inside.
So the real reason may well be "weighted average of other blues with overlapping description", followed (not preceded!) by a language model producing some convincing words that can communicate the mental state to other humans.
IMO, the main difference between AI and humans right now, is that humans can do human things with far fewer examples. AI has the speed advantage that lets them read all the words in the world, listen to all the music ever recorder, watch all the videos ever filmed… but are so stupid[1] that they need to, too.
[0] I do sometimes notice this in my own brain: I can have a fully-formed sentence before my inner dialogue tries to (for a lack of a better word for the internal voice) "speak" it — if I then try to skip the inner voice because I've already had the thought and turning into an imagined voice is clearly a waste of time, the part of me which is the inner voice is, for lack of a better word, frustrated.
[1] Stupid on this particular axis. Intelligence isn't fantastically well defined even in humans, is badly defined cross-species, and doesn't have a singular definition at all for machines, so it is perfectly valid to consider intelligence to be a question of "what can it do?" rather than "how fast does it learn?", for which many current AI (there's no G, so narrow AI that only plays chess is fine) "smarter than humans".
IMO, the main difference between AI and humans right now, is that humans can do human things with far fewer examples. AI has the speed advantage that lets them read all the words in the world, listen to all the music ever recorder, watch all the videos ever filmed… but are so stupid[1] that they need to, too.
I mean, if humans need to listen to far fewer examples to create music than an LLM needs to parse sound files, they're obviously not doing the same thing, are they?
Similarly, it's unambiguously false that a human blues is the weighted average of the sound files of other blues. There are structures which are essentially identical to those used by other artists for reasons of easy improvisation, there are selections of notes played over the top which are a function of fundamental musical properties of consonance and dissonance, scale runs the instrument is suited to as well as licks that may have been directly borrowed, but the final output is dependent heavily on the actual capabilities of the artist to play and singer, which obviously aren't simply lifted from their favourite artists' recordings. Even if it's highly derivative which it almost certainly is, it's not remotely the same progress as a matrix generating a sound file from a combination of other tagged sound files and the input "12 bar blues with Stevie Ray Vaughan guitar work and BB King vocals"
I mean, if humans need to listen to far fewer examples to create music than an LLM needs to parse sound files, they're obviously not doing the same thing, are they?
Similarly, it's unambiguously false that a human blues is the weighted average of the sound files of other blues. There are structures which are essentially identical to those used by other artists for reasons of easy improvisation, there are selections of notes played over the top which are a function of fundamental musical properties of consonance and dissonance, scale runs the instrument is suited to as well as licks that may have been directly borrowed, but the final output is dependent heavily on the actual capabilities of the artist to play and singer, which obviously aren't simply lifted from their favourite artists' recordings. Even if it's highly derivative which it almost certainly is, it's not remotely the same progress as a matrix generating a sound file from a combination of other tagged sound files and the input "12 bar blues with Stevie Ray Vaughan guitar work and BB King vocals"
> I mean, if humans need to listen to far fewer examples to create music than an LLM needs to parse sound files, they're obviously not doing the same thing, are they?
If I need much longer to run a marathon than an athlete, is that because my body is not doing the same things as the athlete, or just because it's doing all the same things but doing them worse?
If a (clinical) moron tries to learn a song, and it takes a lifetime, why? (Do we even understand human brains well enough to answer that? Not my field).
> there are selections of notes played over the top which are a function of fundamental musical properties of consonance and dissonance, scale runs the instrument is suited to as well as licks that may have been directly borrowed
How do you learn what these things are? What do "consonance and dissonance" mean in terms of why they make you feel the way they do, and why can't an AI learn those things from examples? The qualia, that AI hopefully doesn't have but we can't tell because we don't have a test for that?
Danse Macabre (Camille Saint-Saëns) was rejected at the time for how it sounded. The early cultural rejection of Blues seems to me like it was more probably racist excuses than artistic preferences, which is why I'm picking a non-Blues example.
> reasons of easy improvisation … [all the other things in the previous quote, I hope this is a fair cut] … the final output is dependent heavily on the actual capabilities of the artist to play and singer, which obviously aren't simply lifted from their favourite artists' recordings.
If I put the AI into a robot (or simulated) body, and the limits of that body constrained the capabilities of the output, those limit themselves make it OK? Constrain it to MIDI rather than a full waveform?
This doesn't seem like it would change the answer, to me.
If I need much longer to run a marathon than an athlete, is that because my body is not doing the same things as the athlete, or just because it's doing all the same things but doing them worse?
If a (clinical) moron tries to learn a song, and it takes a lifetime, why? (Do we even understand human brains well enough to answer that? Not my field).
> there are selections of notes played over the top which are a function of fundamental musical properties of consonance and dissonance, scale runs the instrument is suited to as well as licks that may have been directly borrowed
How do you learn what these things are? What do "consonance and dissonance" mean in terms of why they make you feel the way they do, and why can't an AI learn those things from examples? The qualia, that AI hopefully doesn't have but we can't tell because we don't have a test for that?
Danse Macabre (Camille Saint-Saëns) was rejected at the time for how it sounded. The early cultural rejection of Blues seems to me like it was more probably racist excuses than artistic preferences, which is why I'm picking a non-Blues example.
> reasons of easy improvisation … [all the other things in the previous quote, I hope this is a fair cut] … the final output is dependent heavily on the actual capabilities of the artist to play and singer, which obviously aren't simply lifted from their favourite artists' recordings.
If I put the AI into a robot (or simulated) body, and the limits of that body constrained the capabilities of the output, those limit themselves make it OK? Constrain it to MIDI rather than a full waveform?
This doesn't seem like it would change the answer, to me.
> If I need much longer to run a marathon than an athlete, is that because my body is not doing the same things as the athlete, or just because it's doing all the same things but doing them worse?
If it's because you have to generate the concept of legs from observing other people run marathons before emitting a digital facismile of a race, I think we can be confident you're not doing the same thing as the other athletes.
How do you learn what these things are? What do "consonance and dissonance" mean in terms of why they make you feel the way they do, and why can't an AI learn those things from examples? The qualia, that AI hopefully doesn't have but we can't tell because we don't have a test for that?
Consonance and dissonance are physical properties of sound resonating in our ears, our emotional reactions involve observable physiological reactions like dopamine release and I think we can confidently reject the premise that a static model of relationships between wav files which instantaneously generates another wav file is doing the same thing
Nobody is arguing musical fashions don't exist, because they do, they're arguing that AI unambiguously does not "listen" to music and a human performer unambiguously isn't even capable of outputting a mathematical translation of all the sound files they've absorbed, never mind solely doing that like a large model trained on other people's music [and text]
If it's because you have to generate the concept of legs from observing other people run marathons before emitting a digital facismile of a race, I think we can be confident you're not doing the same thing as the other athletes.
How do you learn what these things are? What do "consonance and dissonance" mean in terms of why they make you feel the way they do, and why can't an AI learn those things from examples? The qualia, that AI hopefully doesn't have but we can't tell because we don't have a test for that?
Consonance and dissonance are physical properties of sound resonating in our ears, our emotional reactions involve observable physiological reactions like dopamine release and I think we can confidently reject the premise that a static model of relationships between wav files which instantaneously generates another wav file is doing the same thing
Nobody is arguing musical fashions don't exist, because they do, they're arguing that AI unambiguously does not "listen" to music and a human performer unambiguously isn't even capable of outputting a mathematical translation of all the sound files they've absorbed, never mind solely doing that like a large model trained on other people's music [and text]
So, for you, it would change the answer if I put the AI into a robot (or simulated) body, and the limits of that body constrained the capabilities of the output?
> Consonance and dissonance are physical properties of sound resonating in our ears,
If a duff chord is played out of hours in a full morgue where no living person can hear it, is it still dissonant?
> our emotional reactions involve observable physiological reactions like dopamine release and I think we can confidently reject the premise that a static model of relationships between wav files which instantaneously generates another wav file is doing the same thing
I'm not so sure.
Observable physiological reactions are the easy part, but they don't solve the "hard problem of consciousness"[0]. Various chemicals, including dopamine, modulate various neural activities and let our brains focus on things and learn in certain ways. Attention heads allow a big confusing pile of linear algebra to "focus" on particular things and learn in certain ways — what's that like on the inside? Is there even an inside for it to be like something? Nobody knows. I've only heard one suggestion for how to test it that isn't immediately obviously flawed: https://www.youtube.com/watch?v=LWf3szP6L80
(I don't know if Ilya Sutskever came up with this test, but he's who I first heard promoting it).
We're made of atoms. The chemistry of life led to us having feelings as the result of evolution, a process that has no intentionality. An AI may, or may not, reproduce that. In the absence of a test (without which we can't even determine the presence or absence of qualia in mammals with brains of similar complexity as SotA AI models), I'd be just as skeptical of anyone who says some AI does have it as anyone who says that AI doesn't have it. At the same time, given how we got it, I think our human brain chemistry is far less magical and more "mess that works" than our egos want to believe.
I'm happy to call current AI "fast but stupid". Anything else, including future AI, I'm not so sure about.
[0] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
> Consonance and dissonance are physical properties of sound resonating in our ears,
If a duff chord is played out of hours in a full morgue where no living person can hear it, is it still dissonant?
> our emotional reactions involve observable physiological reactions like dopamine release and I think we can confidently reject the premise that a static model of relationships between wav files which instantaneously generates another wav file is doing the same thing
I'm not so sure.
Observable physiological reactions are the easy part, but they don't solve the "hard problem of consciousness"[0]. Various chemicals, including dopamine, modulate various neural activities and let our brains focus on things and learn in certain ways. Attention heads allow a big confusing pile of linear algebra to "focus" on particular things and learn in certain ways — what's that like on the inside? Is there even an inside for it to be like something? Nobody knows. I've only heard one suggestion for how to test it that isn't immediately obviously flawed: https://www.youtube.com/watch?v=LWf3szP6L80
(I don't know if Ilya Sutskever came up with this test, but he's who I first heard promoting it).
We're made of atoms. The chemistry of life led to us having feelings as the result of evolution, a process that has no intentionality. An AI may, or may not, reproduce that. In the absence of a test (without which we can't even determine the presence or absence of qualia in mammals with brains of similar complexity as SotA AI models), I'd be just as skeptical of anyone who says some AI does have it as anyone who says that AI doesn't have it. At the same time, given how we got it, I think our human brain chemistry is far less magical and more "mess that works" than our egos want to believe.
I'm happy to call current AI "fast but stupid". Anything else, including future AI, I'm not so sure about.
[0] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
> If a duff chord is played out of hours in a full morgue where no living person can hear it, is it still dissonant?
Obviously, since we can't prove the morgue isn't consciously listening to it... :p
We can't prove that Napster or a counterfeit CD isn't conscious to everyone's satisfaction either, but fortunately for copyright-holders, that isn't actually how the burden of proof on a "it's just like humans learning how to perform" defence works. Legal systems deny that things that probably are conscious are human-equivalent all the time anyway, that's why you can eat or sell a cow but you can't sue it or assign copyright to it.
But even if the burden of proof was reversed, an argument that an NN consisting of static values stored on disk that outputs a discrete and (similarly) static sound file on demand before returning to being static without even a state update isn't the same thing as a human mind/body constituting long-running chemical processes performing music in continuous time seems.... pretty sound.
Obviously, since we can't prove the morgue isn't consciously listening to it... :p
We can't prove that Napster or a counterfeit CD isn't conscious to everyone's satisfaction either, but fortunately for copyright-holders, that isn't actually how the burden of proof on a "it's just like humans learning how to perform" defence works. Legal systems deny that things that probably are conscious are human-equivalent all the time anyway, that's why you can eat or sell a cow but you can't sue it or assign copyright to it.
But even if the burden of proof was reversed, an argument that an NN consisting of static values stored on disk that outputs a discrete and (similarly) static sound file on demand before returning to being static without even a state update isn't the same thing as a human mind/body constituting long-running chemical processes performing music in continuous time seems.... pretty sound.
LLMs don't listen.
LLMs don't do anything.
People do things with LLMs. People write and run code that creates LLMs.
These words "train", "learn" are ML terms that attempt to explain a concept using an analogy. The LLM isn't a person.
Speaking of the LLM as if it is a person, has the rights of a person, or should be legally treated as a person, is a category error.
People do things with LLMs. People write and run code that creates LLMs.
These words "train", "learn" are ML terms that attempt to explain a concept using an analogy. The LLM isn't a person.
Speaking of the LLM as if it is a person, has the rights of a person, or should be legally treated as a person, is a category error.
Yep, between photographers, graphic artists, and musicians I've already seen a massive number of people clamoring for expanded copyright. The wildest thing is, just a couple years ago these people were some of the most vocal against the copyright system, especially those hit by take downs on youtube and other sites.
Creatives forget the tools AI companies will develop for attribution will be applied on their own works as well, and unwanted attributions might pop up. We don't like to admit how much our own creativity is just derivative from a place we forgotten about. But when they make the attribution microscope, it won't be just AIs getting scrutinized.
We have plenty of content with attribution to train such a system - books, art, scientific papers, media articles, even social network comments, we know author and date. We can index everything by author and search against the index to find influences. We can train "text -> author" and the use it for "new text -> attributions".
But will such a system ultimately enable or block creativity? Even past works stand to be revealed in all their influences. Might be embarrassing. On the other hand generative models will be safe as they can sample around problematic attributions and still complete their tasks. We might end up having to use AI for writing in order to ensure copyright safety. The irony.
"I heard you don't like AI in creative fields so from now on you need to use AI to ensure other creative's ideas are not used by mistake, or risk getting sued."
We have plenty of content with attribution to train such a system - books, art, scientific papers, media articles, even social network comments, we know author and date. We can index everything by author and search against the index to find influences. We can train "text -> author" and the use it for "new text -> attributions".
But will such a system ultimately enable or block creativity? Even past works stand to be revealed in all their influences. Might be embarrassing. On the other hand generative models will be safe as they can sample around problematic attributions and still complete their tasks. We might end up having to use AI for writing in order to ensure copyright safety. The irony.
"I heard you don't like AI in creative fields so from now on you need to use AI to ensure other creative's ideas are not used by mistake, or risk getting sued."
Fascinating thoughts, wish I had more than one upvote to give. You make a great point, this could end up backfiring spectacularly because it may expose how unoriginal a lot of work really is.
Coders will have too as well. Coding is an art.
Half-seriously though, being able to provide tooling that could fund a given library could be endangered if global llms can just suggest things because they have parsed a wide array of code.
Maybe it is a lost fight and the value has to be found somewhere else though. Anything physical like deployment, hosting, that AI cannot do, at least for the foreseeable future.
Maybe it is a lost fight and the value has to be found somewhere else though. Anything physical like deployment, hosting, that AI cannot do, at least for the foreseeable future.
Why should anyone care what the copyrights industry's gripes with technology are? They have time and time again tried to stop innovation by blocking MP3 players[0], DVRs[1], Stream Recorders[2], and now AI, despite copyright existing being to further the arts and sciences, at least in the US.
[0] https://cyber.harvard.edu/property00/MP3/rio.html
[1] https://www.tvtechnology.com/news/appeals-court-sides-with-d...
[2] https://www.eff.org/deeplinks/2020/11/github-reinstates-yout...
[0] https://cyber.harvard.edu/property00/MP3/rio.html
[1] https://www.tvtechnology.com/news/appeals-court-sides-with-d...
[2] https://www.eff.org/deeplinks/2020/11/github-reinstates-yout...
It's clear the industry can't stop it. The only thing they can do is shape it into sth that benefits it. Think Napster -> Spotify.
Unfortunately, the way they've historically done it is to try and block anyone with force until someone came along who actually figured out a peaceful model for both consumers and producers (again, think Spotify). I don't think the AI game will play out any different.
The cool thing is: it's a great place for startups. Big tech will get sued right away before finding product market fit with an AI product, but small startups can survive until they have enough product market fit to fund the defense for the lawsuits.
Unfortunately, the way they've historically done it is to try and block anyone with force until someone came along who actually figured out a peaceful model for both consumers and producers (again, think Spotify). I don't think the AI game will play out any different.
The cool thing is: it's a great place for startups. Big tech will get sued right away before finding product market fit with an AI product, but small startups can survive until they have enough product market fit to fund the defense for the lawsuits.
You don't deserve the downvotes.
But I disagree, I think they can stop it, at least for a long while, because of the enormous expense that it takes to train and run these models (at least for now). Nobody operating in legal gray or black territory could afford to do this. If they use the force of law to basically block legitimate uses, it will slow progress to a snail's pace.
Meanwhile they'll use the tech to stay way ahead in the cat and mouse game, so when somebody does generate something they can block it/censor it right away before it's able to spread. That will be good enough for their purposes.
But I disagree, I think they can stop it, at least for a long while, because of the enormous expense that it takes to train and run these models (at least for now). Nobody operating in legal gray or black territory could afford to do this. If they use the force of law to basically block legitimate uses, it will slow progress to a snail's pace.
Meanwhile they'll use the tech to stay way ahead in the cat and mouse game, so when somebody does generate something they can block it/censor it right away before it's able to spread. That will be good enough for their purposes.
An article so self-indulgent you find yourself longing for AI written ones.
>Grainge sports an impressive pair of aural appendages that move up and down the sides of his head when he’s talking.
I mean come on. 13 paragraphs of this before AI is even mentioned.
The part where UMG's CEO is complaining about how "legitimate artists" deserve more revenue from streams was interesting. He's not actually complaining about AI here but about amateur artists getting as much money from streams as "legitimate artists". "Legitimate" meaning fame and a direct tie to UMG, as if those are the measures of a musician's worth.
>Grainge sports an impressive pair of aural appendages that move up and down the sides of his head when he’s talking.
I mean come on. 13 paragraphs of this before AI is even mentioned.
The part where UMG's CEO is complaining about how "legitimate artists" deserve more revenue from streams was interesting. He's not actually complaining about AI here but about amateur artists getting as much money from streams as "legitimate artists". "Legitimate" meaning fame and a direct tie to UMG, as if those are the measures of a musician's worth.
I was going to say it read like a puff piece about a corporation that makes most of its money from legacy back catalog
AURAL APPENDAGES, four letters: EARS.
Is that right?
Is that right?
SCOT or BELL can also work, especially in a cryptic.
I guess this is a comment about context.
a thing that is added or attached to something larger or more important.
"they treat Scotland as a mere appendage of England"
I guess this is a comment about context.
I've been completely blown away by the quality that I'm getting from Suno.ai
Not all of it is good of course, maybe only 10%, but that 10% is of a higher quality than anything I could have produced myself.
Example: https://app.suno.ai/song/08f43c50-3173-4a27-a0b0-3e8dfb10bde...
Example: https://app.suno.ai/song/08f43c50-3173-4a27-a0b0-3e8dfb10bde...
I've been using suno.ai too, here's the most recent thing I made with it: https://voca.ro/15yurCvoA9yR
The process was a little more involved than just using the raw output of suno.ai, I talked about it elsewhere in the thread: https://news.ycombinator.com/item?id=39188980
The process was a little more involved than just using the raw output of suno.ai, I talked about it elsewhere in the thread: https://news.ycombinator.com/item?id=39188980
Although the actual audio quality seems low on playback (a low bitrate MP3?) the structure and arrangement of the song seem fine.
Yet ... the irony of a song of worship created by an AI surely reverberates through the cosmos like a clap of thunder. Surely?
I'm less interested in the quality of the output and more the motivation of the creator. Was it you? Was it made it good faith? What was its purpose? Was it experimental or intended to inspire?
My interested is genuine and not sardonic.
Yet ... the irony of a song of worship created by an AI surely reverberates through the cosmos like a clap of thunder. Surely?
I'm less interested in the quality of the output and more the motivation of the creator. Was it you? Was it made it good faith? What was its purpose? Was it experimental or intended to inspire?
My interested is genuine and not sardonic.
I was just messing around really and thought that it might be easier to find an audience with Christian music rather than if I tried targeting a more mainstream genre. Those mega churches need to play something, right?
I think the example is very impressive. Fully agree with your reasoning. Most music today already feels a bit cringe and interchangeable to me, and this song fits this genre perfectly. I agree that the Christian church would very much be able to use this. Just don't tell them it's done by an AI; that would kill the whole vibe when the target audience is all about being human :D.
I found it really jarring.
Obviously, it's not perfect, and right now, for the best results, it would have to be re-recorded with a proper singer and instruments. But for some dude who has barely any experience in making music, I think it's amazing.
Also, I'm not a Christian; I just experimented in that area because I assumed that the quality bar would be lower in a more niche music genre. I made a custom gpt to generate the lyrics and structure and then just prompted it with, 'write me a song about jesus but don't use his name.'
If this is where it is now, where a person with no skills can make a song that wouldn't seem too out of place on a soundtrack for a Hallmark Channel movie, imagine where it's going to be in 5 years?
Also, I'm not a Christian; I just experimented in that area because I assumed that the quality bar would be lower in a more niche music genre. I made a custom gpt to generate the lyrics and structure and then just prompted it with, 'write me a song about jesus but don't use his name.'
If this is where it is now, where a person with no skills can make a song that wouldn't seem too out of place on a soundtrack for a Hallmark Channel movie, imagine where it's going to be in 5 years?
In my opinion, you could've written a better song yourself by learning three chords on a guitar and producing something more original. Maybe just a confidence thing?
I'm not sure what people are trying to get out of this process, I'm not judging it per se, but what drives people to sit with generative AI, prompt it, and find satisfaction out of the prompting, I don't really know? Is it just the convenience that is appealing? I do understand that it might be fun to explore with the prompting, but I'm genuinely curious about your perspective?
I studied music for a long time, and what I really really gain satisfaction from is actually understanding my instrument, understanding the theory, and then producing music from that knowledge, but in a way, forgetting all the theory and just being myself with my instruments, with little effort.
Like I cannot imagine getting much satisfaction out of having a computer generate a song for me?
Likewise when using Google translate, I've studied Japanese for a long time, and there is a huge level of satisfaction out of actually having a conversation with someone in their native tongue as opposed to hacking through it with GT.
I'm not sure what people are trying to get out of this process, I'm not judging it per se, but what drives people to sit with generative AI, prompt it, and find satisfaction out of the prompting, I don't really know? Is it just the convenience that is appealing? I do understand that it might be fun to explore with the prompting, but I'm genuinely curious about your perspective?
I studied music for a long time, and what I really really gain satisfaction from is actually understanding my instrument, understanding the theory, and then producing music from that knowledge, but in a way, forgetting all the theory and just being myself with my instruments, with little effort.
Like I cannot imagine getting much satisfaction out of having a computer generate a song for me?
Likewise when using Google translate, I've studied Japanese for a long time, and there is a huge level of satisfaction out of actually having a conversation with someone in their native tongue as opposed to hacking through it with GT.
> Like I cannot imagine getting much satisfaction out of having a computer generate a song for me?
Everybody is different, but also prompt engineering can be a highly creative process. I'm at the same place you are where I get a lot of satisfaction out of sitting down with my guitar and creating something, whereas I really don't like prompting AI. But especially for somebody who doesn't have the music skill (which is a lot harder to acquire than most of us remember) if they are heavily involved in prompting and iterating, they could get immense satisfaction out of the process. If it was just "write me a song about Jesus" and that was the end, I don't think anybody would be satisfied, but it's (at least currently) way more involved than that.
Everybody is different, but also prompt engineering can be a highly creative process. I'm at the same place you are where I get a lot of satisfaction out of sitting down with my guitar and creating something, whereas I really don't like prompting AI. But especially for somebody who doesn't have the music skill (which is a lot harder to acquire than most of us remember) if they are heavily involved in prompting and iterating, they could get immense satisfaction out of the process. If it was just "write me a song about Jesus" and that was the end, I don't think anybody would be satisfied, but it's (at least currently) way more involved than that.
How about a little datascience rap? https://app.suno.ai/song/fbfe344c-467f-47d2-b1a3-2b70cce651d...
What about it?
You know what I'm asking. Is it also jarring, is it funny, is it interesting, etc.
You're asking if I find it jarring ?
All the rap music I like is trying to convey some type of deeper message, or make some type of important social commentary, and rap often delivers that in a powerful, fun and beautiful way. It's not just the rhyming that is impressive, it's the culture that goes with it.
I get that it's kind of cute, and geeky to have the computer use statistics to produce rhyming sentences, but it just will never hit the same way as 2pac, Eminem, ODB, Marvin Gaye or Gill Scot Heron does, not matter how sophisticated it gets.
People can argue all they like about humans "just being LLMs" but humans also bring real life experience, failure, suffering and meaning to their art, which IMO is very important for the art to be interesting and develop connection.
Van Gough is like this, his work is interesting to me because I find his life and his story interesting.
So I don't find that rapping jarring, but I don't care for it either, it does absolutely nothing for me and I'd never listen it a second time, it just sounds like a rip off or approximation of many artists I've heard before.
All the rap music I like is trying to convey some type of deeper message, or make some type of important social commentary, and rap often delivers that in a powerful, fun and beautiful way. It's not just the rhyming that is impressive, it's the culture that goes with it.
I get that it's kind of cute, and geeky to have the computer use statistics to produce rhyming sentences, but it just will never hit the same way as 2pac, Eminem, ODB, Marvin Gaye or Gill Scot Heron does, not matter how sophisticated it gets.
People can argue all they like about humans "just being LLMs" but humans also bring real life experience, failure, suffering and meaning to their art, which IMO is very important for the art to be interesting and develop connection.
Van Gough is like this, his work is interesting to me because I find his life and his story interesting.
So I don't find that rapping jarring, but I don't care for it either, it does absolutely nothing for me and I'd never listen it a second time, it just sounds like a rip off or approximation of many artists I've heard before.
For me, this one is really off. The music is exceptionally good, the vocals are astonishing, but the text doesn't at all fit with hiphop subculture.
Yes, this will benefit record companies.
But it will also benefit indie artists, who'll be able to create record label quality tunes with much less effort, just like Stable Diffusion makes it possible for anyone to be an "artist".
"In April of 2023, an anonymous producer called ghostwriter used A.I. voice replications of Drake and the Weeknd to create a deepfake duet called “Heart on My Sleeve.”
But it will also benefit indie artists, who'll be able to create record label quality tunes with much less effort, just like Stable Diffusion makes it possible for anyone to be an "artist".
"In April of 2023, an anonymous producer called ghostwriter used A.I. voice replications of Drake and the Weeknd to create a deepfake duet called “Heart on My Sleeve.”
> But it will also benefit indie artists, who'll be able to create record label quality tunes with much less effort, just like Stable Diffusion makes it possible for anyone to be an "artist".
I don't think this is such a glowing endorsement of why people make music and produce music. I think AI will do amazing things to improve what we have and speed up the most frustrating parts of production (noise removal, time alignment, retuning, writing harmony parts to existing melody lines. Simply look at companies like iZotope (like RX or Neutron), Hush Pro etc - I know it sounds a little 'faster horses' but I really think AI needs to remove the 'hard' part and not the 'necessary part' of being a musician/artist/audio engineer.
The 'black box' effect of AI that simply 'writes a song' feels less useful and less impactful, that's not what being an 'artist' should be about.
I have a sneaking suspicion that major record labels are likely looking at AI for more boring and partially sinister means like DRM tagging/finger-printing, more advanced content-ID etc, heck maybe even virtual artists that they can spin up in-house so they don't have to pay anyone: the perfect 360 artist, geeze.
I don't think this is such a glowing endorsement of why people make music and produce music. I think AI will do amazing things to improve what we have and speed up the most frustrating parts of production (noise removal, time alignment, retuning, writing harmony parts to existing melody lines. Simply look at companies like iZotope (like RX or Neutron), Hush Pro etc - I know it sounds a little 'faster horses' but I really think AI needs to remove the 'hard' part and not the 'necessary part' of being a musician/artist/audio engineer.
The 'black box' effect of AI that simply 'writes a song' feels less useful and less impactful, that's not what being an 'artist' should be about.
I have a sneaking suspicion that major record labels are likely looking at AI for more boring and partially sinister means like DRM tagging/finger-printing, more advanced content-ID etc, heck maybe even virtual artists that they can spin up in-house so they don't have to pay anyone: the perfect 360 artist, geeze.
> heck maybe even virtual artists that they can spin up in-house so they don't have to pay anyone: the perfect 360 artist, geeze.
Spotify would love to direct the listener just to in house AI music that does not pay royalties to the big labels ;)
Spotify would love to direct the listener just to in house AI music that does not pay royalties to the big labels ;)
Yes, in parts.
However, the majority of money is made with the major acts. Taylor Swift, Ed Sheeran, etc.. They are the main revenue drivers.
Royalty free music improves the bottom line only marginally, which is still substantial for an established company like Spotify where small changes in margins count, but it's not revolutionary to the market.
However, the majority of money is made with the major acts. Taylor Swift, Ed Sheeran, etc.. They are the main revenue drivers.
Royalty free music improves the bottom line only marginally, which is still substantial for an established company like Spotify where small changes in margins count, but it's not revolutionary to the market.
> who'll be able to create record label quality tunes with much less effort, just like Stable Diffusion
I have to ask. Was that cynical sarcasm or did you actually mean what you wrote?
Because "record label quality" can go both ways...
I have to ask. Was that cynical sarcasm or did you actually mean what you wrote?
Because "record label quality" can go both ways...
> But it will also benefit indie artists
Definitely not. Music is a market with too much supply relative to the demand. We have too many indie artists that don't make a living; lowering the barrier of entry will create even more indie artists and further worsen the supply/demand imbalance.
Definitely not. Music is a market with too much supply relative to the demand. We have too many indie artists that don't make a living; lowering the barrier of entry will create even more indie artists and further worsen the supply/demand imbalance.
I couldn't get through the start of the article, this guy might be a nice person, but music has gone downhill quite hard in the last 20 years. If this guy is some type of shepherd for the record industry, then I don't have a lot of faith in the future of commercial music.
Maybe the article got better later on but wow, that first 1/3...
Maybe the article got better later on but wow, that first 1/3...
Funny. A 60+ IPR jihadi trying to copy/claim dated pirate parlance.
No thanks, not enough dimensions. Music about what, when ? Not original - or missing emotional edge ? Check what Jarret had to play about april, think Svensson, with Brüninghaus resonating in your head - no need for artificial 'help'. Some old paintings have more depth and behind than nowadays movies (not minding fake news ilustrations).
AI for muzak ? - this may work.
AI for muzak ? - this may work.
Great article. I think that music artists are still needed for live concerts. For creating music to listen to, not anymore.
Collecting, searching and paying for music is done, soon.
Collecting, searching and paying for music is done, soon.
[deleted]