Show HN: AI audio embeddings - Discover 100M+ iTunes songs with language models(speakmusic.sonophase.com)
speakmusic.sonophase.com
Show HN: AI audio embeddings - Discover 100M+ iTunes songs with language models
https://speakmusic.sonophase.com/
4 comments
loving the results for ambient and electronic music. The similarity feature is super cool!
Seems like the system can get confused by certain terms, "Erhu" for example. I can kind of hear how it's making a decision to get items "close" to "Erhu" if it understands words that are similar.
Seems like the system can get confused by certain terms, "Erhu" for example. I can kind of hear how it's making a decision to get items "close" to "Erhu" if it understands words that are similar.
Looks great, what is your tech stack for building this ?
frontend was created with Vue.js
AI training, audio processing and NLP with PyTorch
got a custom implementation based around HNSW graphs for maintaining the embeddings and scalable search
AI training, audio processing and NLP with PyTorch
got a custom implementation based around HNSW graphs for maintaining the embeddings and scalable search
timbre based searches like "racecar" or "rainstorm" yield very interesting results
We've been working on a project to help us discover new music in a more objective way. It's called Speak Music:
https://speakmusic.sonophase.com/
We’ve trained an AI model to understand the correspondence between music and language. The model combines a machine listening and audio signal processing with transformers for text embeddings. Once trained, we index a huge catalogue of unseen audio, ensuring that the search system can efficiently scale to millions of tracks.
At the moment, our model is optimised for our preferred music; electronic, techno, ambient, dub and relaxing etc. We’re currently fine-tuning to handle all kinds of genres and moods.
Our model enables two types of discovery: (a) natural language search and (b) similarity search.
(a) Speak: search for music using freeform natural language prompts. Describe the mood, aesthetic, texture, setting and context of a track.
(b) Music: discover tracks that are acoustically similar to ANY reference track from Apple Music.
We recently presented at Sonar+D in Barcelona.
We hope you like it. Let us know what you think!