With so many people implementing their own SSD streaming for specific combinations of model+hardware, maybe we should look into upstreaming to antirez/ds4 or llama.cpp...
Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.
As a developer, subscriptions come with a few problems:
- Users cannot open their old projects unless they pay the subscription.
- Lower uptake in general, which affects viral adoption ("my friends use this software, so will I"). This is important if you rely on word-of-mouth instead of ad spend.
- Need to keep shipping features to justify the subscription cost. There is less focus on stability/maintenance.
I'm travelling without my Zen 4 machine, or I could test it. ;) Oh well, Compiler Explorer is enough to look at these microbenchmarks on your own.
These simulations are single core to avoid core-to-core latency. Number of cores isn't relevant unless you want to run independent voices/channels and sum them at the end.
So you start with a very optimistic ~90 GFLOPs of 64-bit FMA on Zen 4. Unfortunately, not all operations are clean multiply-adds. Realistically, you'll need trigonometric functions and LUTs, which are quite slower. Btw, the tradeoff between when to compute vs LUT is very fragile and can change due to a ton of factors (notably integrator algorithm).
Then the data you are operating on won't fit cleanly in AVX-512 registers, requiring spills to L1 cache. Ok, still fast on a modern core.
Of course, the peak theoretical number assumes clean vectorization with double-pumped AVX-512... which also won't happen in practice. Classical DSP will fare better (https://www.youtube.com/watch?v=Ssq0a-YdamM) but SPICE integrators are inherently branchy and divergent. Especially for adaptive integrators, you'll waste a lot of operations trying to "lock in" at the exact time point where the waveform turns a corner. Apple Silicon is better at this messy, branchy code.
So yeah, it's possible-but-hard to hit hard-realtime under these conditions.
If I understand this correctly, you could run this compiler on iPadOS (since it's plain C). But you cannot mark it's output executable, since that would require JIT entitlement.
> I think given how fast computers have become relative to digital audio there is probably a good case to just make any "modular synth" run at 32-bit 480KHz or even 4.8MHz through every stage that could process the audio.
1. It should run at FP64 if you want to preserve filter resonances, etc.
2. At 10x/100x fixed-rate oversampling, even a modern "fast" CPU will have very few cycles per (higher-rate) sample to run the DSP for 1 "module" of the software modular. Forget about interconnected modules, multiple tracks, or polyphony. For this kind of "analog"-style processing, it's better to run adaptive-rate algorithms (think SPICE) instead of wasting compute on unnecessary extra audio samples.