RWKV and Mamba families LLM-s have recurrent blocks instead of attention, and they get really close to performance of Transformer on small scale. What you've probably tried was some outdated tech. The biggest advantage of Transformer is that they are easy to scale, and the scale itself brings some quality to the table.
The main drawback of the legacy models was that they were hard to scale, both due to slow training (sequential processing) and poor stability with depth increase (i.e. poor gradient flow). Modern implementation no longer have these problems and able to scale decently.
>the things that Rust prevents me from handling safely because of its own limitations rather than them being fundamentally unsafe are quite rare, and even with those, they tend to be things that I would be quite likely to mess up in practice if I tried to do them in a language without the same safety rails.
Any language with GC can handle complex links between object, including mutable object. Like Erlang/Elixir, JS, Go, etc. You message implies runtime-less language, but majority of practically employed languages are runtime-full.
>I'm curious how you came to that conclusion. It seems wrong both on a theoretical level (Drop/unwinding/catch_unwind should obviously suffice for at least some cases?) and on a practical level (tokio can recover from worker thread panics just fine?).
Tokio provides crash-resistant synchronization primitives, but it cannot recover complex structures you've been handling — you either need to write panic-handlers that would revert your data to more-or-less manageable state or employ ready-made libs that do it for you. Languages designed for crashes don't require ad-hoc recovering — they just automatically clean up everything. That's what I've called "viable".
I mean you could have written crash recovery in C++, but it's just not viable because of how many things can go wrong.
>Maybe you DON'T need that state to be shared, reference-counted, or heap allocated. Maybe you can refactor your code to get rid of those annoyingly hard to deal with abstractions. And you end up with better, more reliable, likely faster code at the end of it.
That's the point 4 in my article — Rust is horrible for mutable shared state. However, in the modern CPU-based programming mutable shared state is like 70% of all the code, so you cannot just ignore it. It's not that CPU-s have to be like that, it's they hapened to be like that.
>there's a concurrency bug I never would have thought of! I guess all that old code I wrote has bugs that I didn't know about at the time.
Programming languages or libraries that excel at concurrency do not use the Arc<Mutex<T>> nuisance. At least they are not imposing it as a main tool. Having shared mutable state does not mean you directly change cell there, like you would in C/C++. I mean if you have a cyclic graph of connected objects — how the hell are you gonna employ Arc<Mutex<T>> for handling them? What Rust actually does is carving in stone pathologic C/C++ ways of "correct handling of shared mutable state" — whatever it is.
>Rust turns unknown failures in C and C++ into known failures and suddenly the C/C++ people start caring about the failures
I'm actually the one who promotes paranoidal assert-s everywhere. I do agree the original statement from the article is ambiguous, probably should have written something like "memory safety in Rust does not increase reliability".
>The pacemaker argument is complete nonsense, because the pacemaker must keep working even if it crashes. You can forcibly induce crashes into the pacemaker during testing and engineer it to restart fast enough that it hits its timing deadline anyway.
I'm not sure whether there is a deadlock-free modification of Rust — deadlock is not considered an undefined behavior in Rust.
>Not a fact. Particularly in the embedded world, crashing is preferable to malfunctioning, as many embedded devices control things that might hurt people, directly or indirectly.
It really depends on how deeply Turing you mechanism is. By being "Turing" I mean "the behavior is totally dependant on every single bit of previous information". For a reliable system turing-completeness is unacceptable for separate functions i.e. it should produce a correct result in a finite amount of time no matter what hapened in the past. Particulary, that's why modern real-time systems cannot be fit into Turing machine, because Turing machine has no interrupts.
>If a pacemaker suddenly starts firing at 200Hz, telling a victim "but at least it didn't crash" is a weak consolation. A stopping pacemaker is almost always preferable to a malfunctioning one
You almost make an excuse for general unreliability of programs. Mainstream C is unreliable, C++ is unreliable, Rust is unreliable. I can agree that Rust is not less reliable than C/C++, but it is definitely less reliable than some other language e.g. BEAM-based ones. I mean in Rust standard library some time ago I actually read "in these and these conditions the following code will deadlock. But deadlock is not an undefined behavior, so it's ok". The designers of Rust did not really try to support any kind of "recover and continue" way of functioning. Yes, you can catch the panic, but it will irreversibly poison some data.
>I would love to hear what he thinks a good programming language is, because I can easily pick more holes in any other language than he has.
I have a working theory that the classic programming is basically dying out:
https://bykozy.me/blog/llm-s-are-not-smart-rather-programmin...
You just cannot write fast and reliable program neither with C++ nor with Rust, because the fundamental model is broken and nobody's really bothered fixing it.
Why would I do this and why do you think I did? Or do you feel like my english is so bad I'm probably writing ransom note out of pieces of LLM output? I can ensure you I'm good enough at typing and talking in english to not require copy-pasting. Do you have any quotes from the article that you feel like were copy-pasted from LLM?
>Do not misrepresent them and pretend that the Rust developers are incompetent or malicious.
I'm sorry, but I sensirely think the Rust designer did a sloppy work by reimplementing C++ flaws with a new syntax. The history repeats itself, the same pathological mechanism once driving C++ development now drived Rust — I mean large enterprise wanting to change everything without changing nothing, the new tool that would feel like the old tool. They did not really invent some new model, look at std::shared_ptr, std::mutex, move semantic — it was already in C++ before the prototype of Rust. The "share immutables, lock mutables" model was a holy grail of C++ concurrency caused by the way STL worked — and STL is not nearly the only container library in C++.
Okay, what's your take on why exactly Rust compiles slowly?
>Compile speed. Why do people care so much? Use debug for correctness and iterating your code. You're hardly going to change much between runs, and you'll get an incremental compile.
What's the largest Rust-based public codebase so far? Rustc with like 1 million lines of code including all the dependencies in all the languages? Zed IDE has 800k lines. Incremental compilation seems to work fine at this scale, but things become messy above it. Most people prefer to not exceed the limit.
>Mutable shared state: make bad designs hard to write. Mutable shared state is another one of these eternal bug sources. Use channels and pass messages.
People think BEAM (Erlang/Elixir) does not have shared mutable state because the programming model says so, but it actually has some mutable shared state transparently under the hood i.e. implementation details and runtime. BEAM processes all run in a single OS address space, they are not even separate threads but async tasks spread out across OS threads — and still per programming model of the language those are "shared nothing". So in the end BEAM is a very good abstraction on the partially shared mutable state.
And no, in BEAM remote messaging is not nearly the same as local messaging. For example, you don't have built-in supervision trees on remote, and ETS is also local-node-only.
I do agree that hardware model of shared mutable state is inherently problematic, but we don't have other commodity CPU-s yet — you have to handle the shared mutable if you wanna be close to hardware.
>It reads like a lot of critiques of Rust: people haven't spent enough time with it, and are not over the learning curve yet. Of course everything is cumbersome before you're used to it.
Well, at least you confirm that the complexity is real. Don't get my criticism to close to the heart though — I did exaggerated the problems a bit, because someone needs to counteract the fanboys.
>"D supports Ownership and Borrowing, just like Rust. DMD, D's reference compiler, can compile itself in less than 5 seconds, thanks to a fast frontend and fast backend. D is easy to metaprogram through traits and templates. You can make your own JIT in D with dynamicCompile."
Indeed, there are languages that have generics, compile blazingly fast, and still have good runtime performance. Go lang is another good example, although not perfect too. If only Rust designers did a single thing to make it compile fast instead of "eventually".
>Yeah sure, but what compares that gives you similar perf, safety, and language features to Rust?
I've already answered it in the original article — Rust is already here, and better language is not. Still, it will not make me say "it's the best option we have by now" — because it's not nearly the best option.
Performance? Lots of code is cold and not impacting performance much. You just don't need everything written in Rust or C++.
>You called out Zed in the blog post as well but I've not seen the Zed devs expressing regret of using Rust. Is this just an assumption on your part?
I'm kinda saying if I was a Zed dev I would have my pillow wet with tears at night. I know this because I participated in IDE development in C long time ago, and I was drowning in this whole low-level stuff all the time, I just could not do a single feature because I have to handle hundred of those other small things before the feature can work.
>As someone who's written many UI applications with egui and one with GPUI, I've felt some minor pain points but nothing show-stopping.
I have no idea what those applications were and how complex they were, so I cannot comment on it.
>You can throw in a destructor [1]. You just need to mark that destructor noexcept(false). You do get a guaranteed std::terminate if an exception escapes a destructor while unwinding is already underway, though.
Come on, please tell me you don't do this in your code. Formally you are correct, but there are many things in C++ that should have better not existed.
>Why doesn't a similar argument apply to "specially designed" Rust?
Because it would lose most of the Rust properties by that time. C code with fixed memory layout, on the other hand, is ideomatic and was widely employed in the past. It's still being employed in embedded. I'm not saying that you are wrong though, there might be people optimizing Rust for this very purpose, but I'm not aware of such an effort.
>catch_unwind exists for a reason.
>>Or compare it to BEAM that can have processes crash all day long and still continue to work.
>Again, nothing stops you from writing Rust that does the same.
Who's gonna GC the poisoned garbage left in undefined state after the crash? I'm not saying it's impossible — I honestly have no idea whether safe recover is possible in Rust, but from what I know it's rather in middle of "not possible" and "not viable".
>There are two conclusions: 1) If Cloudflare hadn't decided on a proper failure mode for this (i.e. a hardcoded fallback config), the end result would've been the same: a bunch of 500s, and 2) most programs wouldn't have behaved much differently in the case of a failed allocation.
So why do they need Rust then? What advantages does it provide? That was the main point of the article — we all wanted a better language, but got another crappy one instead.
>Yeah until that memory safety issue causes memory corruption in a completely different area of code and suddenly you're wasting time debugging difficult-to-diagnose crashes once they do start to surface.
Some very solid argument here. However, as already implied in my article, you can get most of the guarantees without losing your sanity. Memory-safety-related problems are important, but they are not the sole source of bugs in applications — as developers of Zed found out.
>Doing this the "correct" way in other languages has similar impact? So I'm not sure why Rust forcing you to do the correct thing which causes perf issues is uniquely a Rust issue. Doing this the "just get it done" way in other languages will likely come back to bite you eventually even if it does unblock you temporarily.
It might be counterintuitive, but garbage collectors in multithreaded code can be very efficient. I mean you just spawn lots of objects with random references in between and then eventually GC untangles the mess — it's zero overhead until the GC cycle. Escape analysis and semantic-reach containers can reduce GC work a lot (so you don't have the classical JVM GC problems). More specialized things like RCU and general quescence-state reclamation can be totally pause-less.
It's 200% LLM used in production — like 10 hours of dialogs right before writing the article. I had much more hours of coversations with Rust fanboys and it was mostly a waste of time, they just would not try to negotiate on Rust's weak points. I would definitely not be able to write the article with only human support — it's really sad to conclude that LLM-s are much better assistants because they are neutral and objective.
>I've done significant C++ and Rust and Rust compiles WAY faster than C++ for day-to-day incremental compilation. C++ suffers from the header inclusion problem and modules may as well not exist because you can't practically use them.
It really depends on what you are compiling. I did lots of C/C++ that compiled 50k lines of code in 10 seconds FROM SCRATCH — I doubt you can do it in Rust. To be fair, headers in that project were somewhat optimized for compilation speed (not "hardcore", but "somewhat"). People forgot how fast C/C++ compilation can be without 10 Boost includes in each module.
>This is intentionally over complicated, there is no reason for the Box to be there.
Arc<RwLock<Option<T>>> — sounds good now? Don't get me wrong — C++ can be just as horrible, but Rust made it a rule, you can only write your program like this.
>What does this even mean?
I've already answered above, but I can repeat: there are runtime models that allow crash and recover, there are models that crash and limp. In Rust there is only one model of crash: you just crash.
>Eventually I rewrote my btree on top of Vecs. My node & leaf pointers are now array indices. The result? There is no longer any unsafe code. The code has become significantly simpler and it now runs ~10% faster than it did before, which is shocking to me. I guess bounds checks are cheaper than memory fragmentation on modern computers.
Optimizations are very complex and potentially fragile in Rust, LLVM has to sort through tons of generated IR, so it might be just that native Rust structures are optimized better for compilation. Particulary, Rust is able to optimize out some bound checks.
Do note that binary trees are mostly an obsolete legacy today — they are way too cache-unfriendly. I mean you could have written similar code in C++ using std::vector or std::dequeue and get the bounds checking too.
>Everyone talks about memory safety but IMO rust’s best features are enums, traits, cargo, match expressions and so on
C++20 with concepts mostly reproduce the traits. C++17 with std::variants emulate enum/tagged union. Match is unmatched by C++, that's true.
Cargo is good for as long as there are few packages in there. Large projects already suffer from five versions of serde in one build and dependencies on FFI-connected libs that cargo itself cannot build. I mean look at the NPM nightmare — and they've mostly dodged FFI-s.
On the other hand, lets face it: most of the time security in IT systems is the least priority. I mean after shipping fast, iterate fast, better performance, compatibility, architecture soundness (whatever it is), convenient tests, docs with UML diagram, well-designed interface — and somewhere in a distant drawer on the bottom you may find a note about security issues. It's often times people talk about importance of security after it destroyed the whole business.
We need more of the security. We probably don't need the Rust though.
For example, some time ago Mamba-3B matched performance of Pythia-7B: https://www.reddit.com/r/singularity/comments/18asto2/announ...
The main drawback of the legacy models was that they were hard to scale, both due to slow training (sequential processing) and poor stability with depth increase (i.e. poor gradient flow). Modern implementation no longer have these problems and able to scale decently.