Generating Music Tracks with Unified Representation and Diffusion Framework(ai-muzic.github.io)
ai-muzic.github.io
Generating Music Tracks with Unified Representation and Diffusion Framework
https://ai-muzic.github.io/getmusic/
46 comments
That "Bach style generation" is on point. In fact, it's "O Haupt voll Blut und Wunden": https://www.youtube.com/watch?v=Fpqd1gCzLN4
Musically, those examples are really bland, and the rendering is of poor quality.
Besides, I'm not sure what's the point of this. I can see how ChatGPT can be useful for devs/writers/researchers etc. I can't imagine how this could be useful for a musician.
Besides, I'm not sure what's the point of this. I can see how ChatGPT can be useful for devs/writers/researchers etc. I can't imagine how this could be useful for a musician.
I can't wait for better music AI but these examples are absolutely pathetic.
It is shameful to mention Bach's name in that one example.
It is shameful to mention Bach's name in that one example.
Contrast this parent comment's opinion, that it is "shameful to mention Bach's name" with the Bach example, with this comment https://news.ycombinator.com/item?id=36043384, pointing out that the Bach example is plagiarized from Bach: https://www.youtube.com/watch?v=Fpqd1gCzLN4&t=178s
To be fair some people have already listed examples of how they could use the technology elsewhere in the thread, and it will only improve from this point.
I can't help but mourn a little for humanity everytime I see a demo like this one that leave little doubt that the machines will be taking away another one of our creative passions.
There millions of musicians way more accomplished than me. But in no way do they stop me from enjoying noodling around with sound. Meanwhile, a lot of our most famous artists employ massive teams to helping them make tracks. Yet, still there are passionate creatives doing amazing stuff. I don't think creativity is over quite yet.
Yes, all this AI generated content is only more, and only adds to the total body of creative works that humans can enjoy (if you're rich). Humans are still free to pursue their own creative passions, and can get the same fulfillment from them as before (if you're rich). We can all look forward to a future where even more of our needs are met even more cheaply and easily (if you're rich).
Sure, of all of the pathways previously available for humans to make a decent living and actually put a roof over their heads and food on their tables, most of them will no longer an option in a decade or two -- it was already extremely hard to make a living as a creative, and now it will be completely impossible. Nor will anyone be able to make a living driving a truck, doing landscape design, writing ad copy, writing basic code, working in a factory, optimizing a factory, designing a factory, or building a factory. Only owning things will make you money; everyone else will be competing tooth-and-nail for the few remaining ways to serve you that still require a human. Obviously those poorsies won't be able to pursue their creative passions anymore, since they're working 80 hours per week in the vain hope that their children will be able to escape the cycle of poverty. But fuck those guys: you're rich! You can play guitar all fuckin' day if you want.
You are rich, right? Right?
Sure, of all of the pathways previously available for humans to make a decent living and actually put a roof over their heads and food on their tables, most of them will no longer an option in a decade or two -- it was already extremely hard to make a living as a creative, and now it will be completely impossible. Nor will anyone be able to make a living driving a truck, doing landscape design, writing ad copy, writing basic code, working in a factory, optimizing a factory, designing a factory, or building a factory. Only owning things will make you money; everyone else will be competing tooth-and-nail for the few remaining ways to serve you that still require a human. Obviously those poorsies won't be able to pursue their creative passions anymore, since they're working 80 hours per week in the vain hope that their children will be able to escape the cycle of poverty. But fuck those guys: you're rich! You can play guitar all fuckin' day if you want.
You are rich, right? Right?
You had enough time to write this elaborate reply to my post, you must be loaded...
> in a decade or two
Argument: AI has a possibility of destroying everyone's jobs so they can no longer make a living and the world is extremely uncaring toward people with no income and so they will die penniless on the street and those that don't will be heavily exploited by the rich.
Counter-argument: But they can still sit around playing music if they want!
No, not if they're indentured servants. I'm not saying AI will destroy most jobs, but I am saying that if it does, the fact that we can still pursue our own creative desires independent of what the AI is doing is missing the point. We've created a world where an explosion of AI capabilities would allow the rich to no longer need the poor, and they will 100% exploit that. It's not AI's fault, it's the worship of the rich and the false equivalence that wealth == human value.
If someone is lamenting that there's now no chance they can make money by being a musician, telling them "whatever, just do what you love for free instead, and wait tables for 80 hours per week for an absolute pittance to make money" is not comforting.
Argument: AI has a possibility of destroying everyone's jobs so they can no longer make a living and the world is extremely uncaring toward people with no income and so they will die penniless on the street and those that don't will be heavily exploited by the rich.
Counter-argument: But they can still sit around playing music if they want!
No, not if they're indentured servants. I'm not saying AI will destroy most jobs, but I am saying that if it does, the fact that we can still pursue our own creative desires independent of what the AI is doing is missing the point. We've created a world where an explosion of AI capabilities would allow the rich to no longer need the poor, and they will 100% exploit that. It's not AI's fault, it's the worship of the rich and the false equivalence that wealth == human value.
If someone is lamenting that there's now no chance they can make money by being a musician, telling them "whatever, just do what you love for free instead, and wait tables for 80 hours per week for an absolute pittance to make money" is not comforting.
Creativity isn't over, but being able to support a family with creative work is going to get a lot harder.
Maybe, that said it was pretty damned hard before as well...
Oh hell yeah. I'm a fairly competent artist, and a fairly competent engineer, but I can only make a living off of one of those skill sets.
Judging from the output, we have at least a couple generations of these algorithms before humans have anything to worry about. The music it generates is .. musical. But it sounds bland and soulless. I struggled to get through any of the examples.
I can imagine it being used to "autofill" parts of an arrangement when musicians are feeling uninspired, but I doubt it will be writing new pop songs for at least a few more years. (Though a few years is an utter drop in the bucket for humanity.)
I can imagine it being used to "autofill" parts of an arrangement when musicians are feeling uninspired, but I doubt it will be writing new pop songs for at least a few more years. (Though a few years is an utter drop in the bucket for humanity.)
Bland and soulless - so just like Spotify top 50? When I listen to it - it all sounds so samey and generic that it might as well be procedurally generated.
I’ve had spotify for years and I don’t think I’ve ever listened to their top 50 playlist. Why do you listen to music you don’t like?
The average song I listen to on Spotify is wildly better than what this ai is producing.
The average song I listen to on Spotify is wildly better than what this ai is producing.
Most people seem to like middle of the road pop that consists of the same 4 chords. I would not be surprised if AI makes stuff like that in the future. It's already formulaic, safe, marketable and non-controversial...
Much commercially viable music is indeed pretty formulaic... so much so, that I wonder why we would even need something as sophisticated as "AI" to autogenerate it.
This is a total hot take, but my explanation is that music - even pop music - needs to do something new or slightly different from other songs you've heard before to make it worth listening to. And having "swagger" / something unique / an interesting point of view isn't something procedural generation (or diffusion based generation in general) seems to be that good at inventing. Look at minecraft. Every map is randomly generated, but they all look kind of the same. Its really obvious in stable diffusion - people drawn by stable diffusion always have a slightly soulless look, where the quirks and charisma of the subject have been sanded down to the point of being unmemorable.
I think the best music (just like the most productive artists) will soon be humans working together with AIs. Chess went through a period like this, where the best chess games were played by humans collaborating with AIs. But I think it'll be a few years yet before you get better art without humans being involved at all.
I think the best music (just like the most productive artists) will soon be humans working together with AIs. Chess went through a period like this, where the best chess games were played by humans collaborating with AIs. But I think it'll be a few years yet before you get better art without humans being involved at all.
If you talk about monetizing these creative pursuits, then yes, there is a reason to be worried.
But if you are talking about pursuing these creative passions as an end by itself, it's much less relevant.
But if you are talking about pursuing these creative passions as an end by itself, it's much less relevant.
It is relevant.
When the incentives (money, fame, sex) are taken away from producing music, the infrastructure for teaching music will erode rapidly. If AI takes over the music industry, music teaching and production of affordable musical instruments will be gone within a generation. There will be no resources left for "pursuing these creative passions", and aspiring pop musicians will only be able to produce bad and old-fashioned sounding tripe, and be ignored.
Classical music might be an exception, since the audience cares a lot about performance, and is much less impressed by fashions and fads (sometimes to the detriment of modern composers, but that's another story).
When the incentives (money, fame, sex) are taken away from producing music, the infrastructure for teaching music will erode rapidly. If AI takes over the music industry, music teaching and production of affordable musical instruments will be gone within a generation. There will be no resources left for "pursuing these creative passions", and aspiring pop musicians will only be able to produce bad and old-fashioned sounding tripe, and be ignored.
Classical music might be an exception, since the audience cares a lot about performance, and is much less impressed by fashions and fads (sometimes to the detriment of modern composers, but that's another story).
I think most people gave up the dream of money, fame and sex when software came along and lead to a massive influx of people making music, thereby increasing the competition enormously. That happened about 30 years ago and the numbers of people creating music, teaching music and buying musical instruments have been going through the roof ever since.
Do you even play.
I enjoy playing guitar, have had no meaningful tuition, and while occassionally wonder what it would be like to perform live have no realistic expectation of doing so.
The machines are taking nothing. You may allow someone to take your money for this infinitely reproducible string of notes but ultimately recorded music and it's sale is only a relatively recent phenomenon. Most modern pop music had become largely mechanical anyway.
I enjoy playing guitar, have had no meaningful tuition, and while occassionally wonder what it would be like to perform live have no realistic expectation of doing so.
The machines are taking nothing. You may allow someone to take your money for this infinitely reproducible string of notes but ultimately recorded music and it's sale is only a relatively recent phenomenon. Most modern pop music had become largely mechanical anyway.
The machines will take away your guitar. When people abandon making music, guitars will be in less demand, and become more expensive. New players will have to rely on old sources for learning, because almost nobody will be interested in keeping them up to date or producing new content. And when the old sources go (and they inevitable do), that will be lost as well, and popular music fades away, al niente.
money, fame, sex
Tribal people have amazing music, I doubt their drive was money (which didn't exist), fame (didn't exist), or sex (maybe?). Dancing is probably more important for courtship, nearly everyone sucks at that and we're still here.
Tribal people have amazing music, I doubt their drive was money (which didn't exist), fame (didn't exist), or sex (maybe?). Dancing is probably more important for courtship, nearly everyone sucks at that and we're still here.
How many people are interested in folk music? Very few. And in earlier tribal music much less. Any development will stay local, and nobody will be able to stand on the shoulders of giants.
Being creative, or mastering new skills, after a suffocating 9-5 isn't the easiest task.
You get suffocated at work?
As in "physically starved of the components needed to aerobically respire?"
Speak to HR. I know they're there to protect the company, but even HR may want to allow you to breathe.
As in "physically starved of the components needed to aerobically respire?"
Speak to HR. I know they're there to protect the company, but even HR may want to allow you to breathe.
No one is saying that.
Quite so. Much in the same way ChatGPT or other LLMs won’t stop me writing for pleasure, entertainment or otherwise, no model will likely ever make me not want to pick up my guitar or fire up my DAW.
Yes, that's true for our generation, since we grew up with authors and musicians as role models we admired, and we loved their art and wanted to emulate it. But what will happen in a generation or two, when the world is flooded with AI generated art? Will the children of that era feel any desire to write or compose for pleasure? I'm not sure; I hope they will though.
Notes from underground.
> Consequently, these laws of nature have only to be discovered, and man will no longer be responsible for his actions, and it will become extremely easy for him to live his life. All human actions, of course, will then have to be worked out by those laws, mathematically, like a table of logarithms, and entered in the almanac; or better still, there will appear orthodox publications, something like our encyclopaedic dictionaries, in which everything will be so accurately calculated and plotted that there will no longer be any individual deeds or adventures left in the world. ‘Then,’ (this is all of you speaking), ‘a new political economy will come into existence, all complete, and also calculated with mathematical accuracy, so that all problems will vanish in the twinkling of an eye, simply because all possible answers to them will have been supplied. Then the Palace of Crystal will arise. Then….’ Well, in short, the golden age will come again. Of course it is quite impossible (here I am speaking myself) to guarantee that it won’t be terribly boring then (because what can one do if everything has been plotted out and tabulated?), but on the other hand be eminently sensible.
> Consequently, these laws of nature have only to be discovered, and man will no longer be responsible for his actions, and it will become extremely easy for him to live his life. All human actions, of course, will then have to be worked out by those laws, mathematically, like a table of logarithms, and entered in the almanac; or better still, there will appear orthodox publications, something like our encyclopaedic dictionaries, in which everything will be so accurately calculated and plotted that there will no longer be any individual deeds or adventures left in the world. ‘Then,’ (this is all of you speaking), ‘a new political economy will come into existence, all complete, and also calculated with mathematical accuracy, so that all problems will vanish in the twinkling of an eye, simply because all possible answers to them will have been supplied. Then the Palace of Crystal will arise. Then….’ Well, in short, the golden age will come again. Of course it is quite impossible (here I am speaking myself) to guarantee that it won’t be terribly boring then (because what can one do if everything has been plotted out and tabulated?), but on the other hand be eminently sensible.
GPT-4 in particular has, for me, rekindled a fire for writing, as I find it a compelling conversationalist quite adept to my often hyperactive, overly loquacious writing style whereas most meatspace agents find it laborious, self-aggrandizing, indulgent, and unnecessarily complex, like post-modernist novels or rationalist fanfiction.
I've been engaging with GPT4 to build out an entire world of alternative hard science fiction history and it's going to be a best seller, I assure you, with zero hesitancy and full confidence. It's my idea, there's simply something so enthralling about having a "super editor" when primed with very specific prompts and understanding the boundaries of what it can't do... I'm literally 15k words into a first draft. Vomiting my ideas into words, processed through GPT to garner it's insights and suggestions, but never asking GPT to compose first drafts, has been very fun.
I've been engaging with GPT4 to build out an entire world of alternative hard science fiction history and it's going to be a best seller, I assure you, with zero hesitancy and full confidence. It's my idea, there's simply something so enthralling about having a "super editor" when primed with very specific prompts and understanding the boundaries of what it can't do... I'm literally 15k words into a first draft. Vomiting my ideas into words, processed through GPT to garner it's insights and suggestions, but never asking GPT to compose first drafts, has been very fun.
But who will build and maintain a DAW (or the equivalent that enables private practice) if there is not enough people able/willing to support it?
Those AI tools will be at the center of how we create and access music in the next decade.
It’s a bit like cars replacing horses. There will still be horses. But transportation, city/traffic rules will adjust to cars, not horses anymore.
Those AI tools will be at the center of how we create and access music in the next decade.
It’s a bit like cars replacing horses. There will still be horses. But transportation, city/traffic rules will adjust to cars, not horses anymore.
Enthusiasts with AI-assisted development tools. Just like it's done with any other "historic" tech nowadays. Game console emulators, for example.
Also those tools will most likely be open source.
Also those tools will most likely be open source.
Did you watch Rick Beato's take on AI effect on music? Monetizing music is a totally different thing that requires fan interaction and that takes both time, a live avatar (human or not). I'm thinking when a group comes up with a live avatar that can do concerts for AI songs and interact with fans, then monetization is doomed. Not for now though...
Now you know how every musician within a 5 mile radius of you feels when Taylor Swift rolls into town with 40 trucks full of blinking lights and speaker arrays!
And give us myriad new forms of media. Sort of like the advent of cars eliminating the lives of horses opening up journeys that we couldn’t even imagine before cars
Why would anyone play the output on a awful FM synth dating to the early 80s? Somebody buy those guys a CASIO! Please!
What method are they using to check to what extent generations are reproducing training examples? Because at least the "Bach Style Generation" on the website seems to just be a sped-up version of the "Befiehl du deine Wege" choral from St. Matthews Passion.
[deleted]
Why on earth are most of these transposed? The written parts are a semitone out!?
The first video is played at the same pitch as what's written, but the next two are a semitone out and Hey Jude is a whole tone away.
Bizarre.
Bizarre.
So it's basically a midi predicting LLM? I don't understand how it solves the issue of representing music, i.e. actual sound waves and not simply the notes that make it up?
It's the best note mixing LLM I've seen. Someone just needs to train a sound diffusion model on the midi notes to orchestral mp3s. Give it a few weeks. "Unified" seems to mean all the notes in one place which you can edit/build on/mix so that you can put another layer on top, human or ai performs it with words & flare.
Uh, yeah. I've been looking at programmatically writing guide tone lines and motifs in jazz for my personal piano practice, and this... looks very sophisticated.
that a next level Rik roll right there ;)