I've been going smaller.. I have a custom-quantized Rust port of DiffusionGemma (26B) that seems to perform better (in responses) than benchmarks seemed to indicate and reasonably fast for its model size. Works really well on a 36GB mac as well for both prefill and generation.
It's been interesting learning about the balance of factors for performant metal kernels on unified memory.
Should have a repo up on github in the next few weeks.
Author here - if anyone has any contacts at Cloudflare to get the proxied domains (at least roadpay[.]cc) taken down, that would be great. I wasn't able to get an abuse report to stick. Ditto for the related LinkedIn profile and Twitter accounts.
The C2 IP (89.124.107.161) and malware-serving git repo (144.124.244.92) are both hosted on VDSINA in Russia, so not sure if there's anything to do there.
But I think the point is that they should be cheap to set up, and because of the short life, never really contain anything except the potential to compute when needed, not important data.
Om Malik was the guy who had the biggest influence on the direction of my life, by far. It was through him I met Naval Ravikant in 2007, and then through Naval I met my co-founder that led to my startup exit in the '10s.
Luck surface area. I really owe so much to Om. I really can't imagine where I would be without that chance.
One of the most fascinating things to fall out of the AI apocalypse is seeing how abundant AI access amplifies the qualities of a company.
IMO, Deno has always been more methodical, more focused (maybe too focused?) on standards. But now the Deno team is on the right track: using Claude extensively to improve the node.js compat which was absolutely herculean if not impossible before AI. [+]
On the other hand, Bun has always played a bit fast and loose, chasing metrics at the cost of stability. Access to abundant AI has sent that project off the rails.
Disclaimer: Former Deno engineer - I'm obviously going to have some biases. All IMO of course, but if you ask me I'd still bet on Deno in the long term, and I personally still use it for any .ts projects.
[+] There might be a dozen people in the world that know how sensitive and subtle the timing and ordering in the JS event loop is and how meticulous just this single part needs to be for major node.js projects not to completely crap themselves.
I'd love it if WASI modules could introspect their own custom sections (potentially even more introspection than that), but I've never been able to figure out a good way to do this. Seems like a fairly useful feature for a few use cases.
Author here, happy to answer any questions. I've been working on building some higher-level abstractions on link sections (specifically, link-time optimized collections like maps (1) and sorted slices (2)) and wanted to share the hard-fought knowledge from the last couple of months.
There's a decent amount of knowledge around pre-main work in Rust, but I think this is one of the first attempts to walk through mutable link sections, which open up a pretty wide world of optimization, IMO. Even without mutability, I figured there isn't nearly enough documentation on these approaches out there.
Types and generics are hard to parse in regular macros without a tt muncher. Ditto for fn args. If you need to do actual matching of types, you can't capture them as $xxx:ty because Rust will not allow a larger matched token to be broken up again (unless you use a paste! hack to roundtrip it back into ungrouped tokens).
I think I'd feel more comfortable if I could drop an .rs file into the eBPF dir instead of a .c one. It's already a Rust project! :)
And for some reason I was expecting this to be a kernel-accelerated webserver - if that could be done safely using eBPF that would be amazing!
Also, single-threaded? Forking and sharing an incoming connection queue is basically trivial on Linux, that should be literally just a few lines, even with Rust. Use SO_REUSEPORT and the kernel will do the rest.
FWIW, if you're going to push for io_uring, you should also be pushing kTLS IMO, you'll drastically simplify your design if you can avoid pumping userspace SSL after the handshake.
As someone who has spent a _lot_ of time writing declarative and procedural macros, the important thing to ask before digging into a macro is whether you need a procedural macro at all.
Complex proc macros absolutely do slow builds down. In many cases, a proc macro only need to be a stub that can delegate to a declarative macro.
You may not need to use syn/quote, but if you are doing any sort of processing/parsing of Rust code you pretty much need to.
FWIW, I really hope that the Rust project focused on finer-grained token matching in declarative macros so we can migrate most proc_macro code away. The macro system is powerful, but nowhere near where it needs to be.
I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
I've weirdly found that I like the Google AI mode in specific cases, and I find that the hybrid is the worst of the two worlds. There are some cases where I don't know exactly what I'm looking for and I want the AI to curate results. In other cases, I know what I'm looking for and I want to read the OG source.
The AI popup is the worst and will hallucinate answers from Reddit comments. I specifically had it ask me a nonsense question which was literally just someone's Reddit comment suggesting a follow-on topic B to the search topic A. The AI mode will _sometimes_ be useful enough to prompt into doing the search and summarization for me and get me just enough info and some links to continue the work myself.
The most effective rule for me is no addictive apps on my phone or laptop - browser apps only. The browser apps are _far_ less addictive and just enough friction to keep me off them for extended periods of time. As well, infinite scroll just isn't as effective in a browser and there's a real feeling of limited content running out.
Playing with fun tech.