Master's in CS from UPenn
Bachelor's in Math from Towson
Five YOE developing financial models, doing data analysis, and assessing risk at KKR and Transamerica
Currently a teaching assistant at UPenn for machine learning and object-oriented programming, and a course development assistant for web development.
To do this, I think you are right that you would need to 'downgrade' modern recording to sound old so that you have both side of the training data covered.
This would be a cool project to work on. Ideally you would buy some vintage gear and then run the audio through both, but that would be very expensive. You could may be find some vst emulations though and get decent results.
Interesting! I had not seen this. On their website they mention diffusion but not the other models so it might not be identical but its definitely similar.
I sound design a lot of stuff (in fact I made some of the default kicks in the app), but this is just a different tool, and I wanted some practice training and deploying a generative AI model.
Seems like if you build some more scaffolding around it, it wouldn't be bad. I think AI video isn't quite there yet so you probably would want to lean into that. For example you could ask for an animated or cartoon music video so the real shots don't look weird. Also if you gave it some guidance on what a good music video is like it would probably help as well. But yeah idk may be that's not the goal here.
If you are committed the model should work about the same on any type of one shot sample. The code is public and documented so if you have the snare collection and a macbook you could probably point claude/chatgpt at it and it would be able to train on your laptop.
SUNO is pretty close. It still has some weird things going on with high frequency artifacts and phase between left and right channels but if you aren't listening on a good system (like a phone) most people probably wont notice.
The compression is the OTT which stands for Over The Top compression. It was originally a multiband compressor preset in ableton and is now used widely throughout dance music.
the spectrograms are 128x173 (128 mel frequency bins by 173 time frames)
the encoder is downsampling 4 stages of stride 2 convolutions so it halves dimensions 4 times
0: 128 x 173
1: 64 x 87
2: 32 x 44
3: 16 x 22
4: 8 x 11
Then i used 4 separate channels.
This was somewhat arbitrary due to the local training constraint. This would be a hyper parameter worth tuning if I had time to dig into this more.
I trained this a few month ago and don't remember exactly what I tried before I arrived here, but I only ran the whole process 2 or 3 times because of how long it took to train.
Hope this answers your question!
If you are curious I used a NVIDIA GeForce GTX 1660 SUPER
So to be exact, it came out 7 years ago (I upgraded at some point on this desktop a long time ago and didn't remember the exact year) (I updated the article to reflect this now)
This cost $230 new and you can get one now for $100 which I don't think is too out of reach.
I wouldn't exactly say it's trying to solve a problem. It's to explore and see what happens which is what music is all about. It's also a unique niche model I haven't seen before.