i've spent the last couple years experimenting with LLMs that write procedural, ambient music: instead of generating an audio file, we're using systems that write actual source code (supercollider synths) so the music runs live and can be embedded in and controlled in real time by software like games. a surprising amount of the work is taste -- prompt engineering and curating an aesthetic so the generated code reliably compiles and sounds good, with a compiler and automatic repair in the agent loop when it doesn't build. today, it produces single-synth musical systems, but I'm researching how to orchestrate multiple synths into fuller, evolving compositions with deep musical form and structure. LLMs make algorithmic music far more accessible to artists, and I'm eager to hear the new kinds of music that come out of it. hit me up if you're an audiophile interested in collaborating.
a little suite of word games I've been building as the testbed for Underscore. every puzzle is generative: crosswords with LLM-written clues, a word ladder, a guess-as-it-draws game, and a deduction game scored on how few clues you need. the crossword grids come out of a Rust constraint-satisfaction solver — backtracking fill over a bitmask-indexed word list, enforcing 180-degree rotational symmetry and slot intersections. each generated puzzle gets its own generated, continuously-evolving soundtrack with no audio files involved. right now, the music is generated per puzzle, but it needs some work. it's currently too drone-y, and needs to be more magical/whimsical. where I want to take it is having it respond to the gameplay itself — which is really the whole point of Underscore
I’m working on Underscore: a web SDK and browser sandbox for making procedural music systems with LLMs.
The basic idea is "music with source code." Instead of prompting for finished audio files, you use an LLM to help write and revise a SuperCollider-based system that runs in the browser via WebAssembly [1]. The result is executable music: inspectable, editable, versionable, and controllable at runtime.
I’m especially interested in adaptive sound for software: games, creative tools, meditation apps, AI agents, interactive art. Places where a static audio file feels too dead, but hiring a composer/sound designer for every variation is unrealistic.
It’s early, but the thesis is that LLMs make algorithmic music much more approachable because code becomes a conversational medium. I wrote a longer piece about the idea here: https://x.com/osetinsky/status/2053674503801028944?s=20
- Sam Aaron for building SuperSonic, allowing for SuperCollider in the browser as an AudioWorklet: https://sonic-pi.net/supersonic/demo.html. Earlier, pre-LLM versions of Underscore relied on low-latency WebRTC implementations for streaming SC synths running on servers to browsers in real-time