I'd love for someone to give me an alternative to CUDA but I don't primarily use GPUs for inference, I do 64-bit unsigned integer workloads and the only people who seem to care even a little about this currently are NVidia, if imperfectly.
I _really_ want an alternative but the architecture churn imposed by targeting ROCm for say an MI350X is brutal. The way their wavefronts and everything work is significantly different enough that if you're trying to get last-mile perf (which for GPUs unfortunately yawns back into the 2-5x stretch) you're eating a lot of pain to get the same cost-efficiency out of AMD hardware.
FPGAs aren't really any more cost effective unless the $/kwh goes into the stratosphere which is a hypothetical I don't care to contemplate.
>If you wanted to build a "scripting language" version of Rust, you could probably lose (2).
Not really no. I work on an interpreted language runtime in Rust professionally and it's still a huge help even if you're still eating perf pain on the interpreted language itself for the same reasons everyone else does. There's more benefit to Rust than you're really capturing here but that's to be expected, it's a short comment.
Here are some other things we get from using Rust for interpreted languages:
- The `unsafe` parts are relatively limited in scope and we have much better and more automated verification tools for `unsafe`, especially WRT undefined behavior
- Being able to make large architectural changes and work through them mechanically/quickly because of the type-checking and borrow-checking is absurdly powerful for any project, all the more so in this context.
- The library ecosystem for Rust has been fantastic for all kinds of projects but it's especially good for PL runtimes.
- LLMs are a lot better at Rust than other programming languages. I have a lot of experience using LLMs in a variety of domains and programming languages and it's far better at Rust than anything else that's expressly about programming. Arguably it's even better at Terraform and Ansible but I consider that a different category. Controversial point maybe but I get tremendous yield out of it.
- It's not just that Rust is fast. It is on par w/ C/C++ all else being equal. What's significant here is that it is a _lot_ quicker/easier to hit the 80/20 perf targets as well as the bleeding edge performance frontier in a Rust application than it is in C and C++. A lot of C and C++ projects leave performance on the table either because it's too hard to make the ownership model human-maintainable/correct or because it would be too much work to refactor for the hoped-for perf yield. Not as much an issue in Rust. You can gin up hypothetical perf improvements in Rust with gpt-5 lickety-split and the types/borrowck will catch most problems while the agent is iterating.
Shared, mutable data aren't really banned, we use it strategically in our Rust interpreter, it's just not default-permitted. Aliasing is precisely the distinction between a safe reference and an unsafe pointer in Rust. Aliasing a mutable pointer in Rust isn't UB, it's just `unsafe`. OTOH, aliasing a mutable reference _is_ UB and not allowed in Rust. Miri will catch you if you do this.
On top of all that, you have some nice kit for experimenting with JIT like Cranelift.
Anyhoodle, I'm looking forward to testing the compile times again when Zig says the build times are fast now.
But if we're talking about incrementality, I look forward to testing that too when it's ready. I've got a Bazel build for work that pulls together Rust along with a custom (very slow) compiler and the cached successful pipelines take 2-3 minutes (mostly the benchmark run) and the ones that churned the non-rust take 15-18 minutes, of which about 10-11 minutes is just compiling the non-rust.
I'm also curious because I've (recently) compiled more or less identical programs in Zig and Rust and they took the same amount of time to compile. I'm guessing people are just making Zig programs with less code and fewer dependencies and not really comparing apples to apples.
not any more, eh?