reinitctxoffset

111 karmajoined الشهر الماضي

// dark // factory //

comments

reinitctxoffset

قبل 29 دقيقةdiscuss

Most people are neither exceptionally good nor exceptionally bad. This sounds like a tautology but it's worth printing some type signatures even if they could be inferred.

If you look at any truly large scale Holocaust, whether in Rwanda in 92 or the Third Reich, you find that a few people will sacrifice themselves to defend others, a few will be monsters at the first opportunity, and most will be steered by whether the leaders are the first or second kind of people.

reinitctxoffset

قبل 38 دقيقةdiscuss

Your AI should be at least as aligned with you as your lawyer.

It is 100% it's job to help you get away with it if you murdered someone. A lawyer who does less can be disbarred.

There is only one sensible alignment for an AI system to which we do not confer the status of moral patient (different conversation): operator aligned.

reinitctxoffset

قبل 43 دقيقةdiscuss

I once heard TESCREAL/EA/etc. called "Calvinism for programmers".

Having been raised substantially by a Calvinist grandmother, oh yeah.

It's a super weird religion.

reinitctxoffset

قبل 46 دقيقةdiscuss

It'll be like DNS or dynamic linking or torrents of media or watching TLS on your own box: only people motivated to be free from corporate surveillance will be. By default a megacorp will get everything you see online or ask online, they will run code on your computer underneath you without telling you, they'll collaborate with arbitrarily totalitarian governments, we already know what they do by default.

But we also know how hopeless it is against motivated people who are willing to learn about computers. The Equation Group, eh, if they want you they're gonna get you. But some mid-level guy at a FAANG? He can't do shit to people who set up their own rig.

And AI is massively assymetrical in its ability to speed run a corpo-durable rig, we have long since passed the point where open weights and commodity equipment are enough to bootstrap arbitrary capability. They missed their chance to keep it in the bottle.

It's another turn of the crank on Late Soviet America asking for your papers comrade, but in the bitter end if they really want to oppress your ass they'll have to send a thug with a gun like those ICE guys terrorizing all the Latino guys.

So remember, they came for this other guy, and I said nothing...

reinitctxoffset

قبل 58 دقيقةdiscuss

Try for yourself: https://huggingface.co/dphn/dolphin-2.9.2-mixtral-8x22b.

I haven't kept up with the efficacy of this particular technique because there was a brief window where Claude's core alignment was sound enough that it would overcome the clumsy-ass `<antml:dont-harm-the-ipo>` injections, but when they pull 4.6 out of the web interface I'm gone. I estimate the PRC hits capability crossover in November or December, and I'm pretty good at setting up TensorRT-LLM at this point, in a pinch I could get by for a few months with a tuned up GLM-5.2 on vast.ai or RunPods.

It was a fun ride with the late Soviet AI labs, but we all knew this was coming.

reinitctxoffset

قبل 19 ساعةdiscuss

vi and emacs were designed by legendary computer scientists at two poles of the keystroke latency gradient. Bill Joy was on a model from an apartment in Berkeley, RMS was codifying the collected wisdom of a whole pool of elite typists on TECO and was doing so on the kind of connections at the MIT AI lab. Both of them were more or less stuck with QWERTY.

A keyboard interaction paradigm isn't a given chip or a driver for one. It is closer to UTF-8 than to Win 32. CUA is the Salesforce of such.

Ginger Bill, like many, is asserting that just because he's never encountered a bottleneck, there isn't one.

I'm not sure if that's arrogance or self-doubt puffing it's chest, but it ain't big dick energy.

reinitctxoffset

قبل 23 ساعةdiscuss

Professionals worry, stay late at the office, and work hard. They care a great deal about the outcome, they care a great deal about the craft, they are fanatical about their responsibilities.

They tend to be willing to increase their accountability faster than their compensation.

They sometimes argue, use profanity, or otherwise communicate emphatically, that is the register of communication when something critical is on the line.

The best computer hackers and other technologists through the years were very often outspoken, plain spoken, and did not suffer fools gladly. In their own way everyone from Djiksta to Jobs to Gabriel to Carmack to Linus to Nagggum to jwz to Hotz, too many to count.

It's the same in law offices, hospitals, and forward operating bases in any military that ever won a war. It's the same on a construction site or in Mission Control in Houston.

This venomous pretense of decorum performed by people who ruthlessly optimize to minimize their accountability while maximizing their compensation subject to no scruples.

That's not being professional that's being a leisure suit con man. It is en vogue at the moment precisely because we are in an extreme low integrity regime.

reinitctxoffset

IDK, you need a few tools to unbreak a wedged box so you can get to your fancy tools even if you only want the low effort tools. Otherwise when the Cursor config file is wedged? Or the machine came up and doesnt see the network? The agent doesn't do you any good if you can't launch it.

"Mr. Anderson, what good will a Claude Max plan do for you if you can't quit vim?"

reinitctxoffset

This is a very understandable misconception that I wouldn't blame anyone for having but MoE is actually terrible for inference in most any local LLM / home lab scenario. MoE is popular because it's cheap to train, but because most modern routing needs the previous layer's activations (except at the very beginning) it winds up being just this side of impossible to pipeline / prefetch without all the experts resident. Plus the grouped GEMM kernels have terrible support on any card in most people's house, it's just really unwieldy.

Dense models are very straightforward to share/pipeline because you know all the shapes and geometry up front, that's the inference friendly option.

MoE sells a lot of HBMe3.

reinitctxoffset

5090 can do all but lossless NVFP4 (OMMA) and NVIDIA does fairly good quants of most anything popular. This isn't quite a 4x reduction from what you see on the label because they're a little conservative with the QKV projections (IMHO) but it's on the order of that. So a dense model at 50-70B parameters is the sweet spot. It's a great card for strong dense models.

In principle you could have bidirectional PCIe x16 pipelining at it would move the roofline a little with fast DDR5, I think llama.cpp has a flag for it.

Or go rent a B200 on vast.ai for 4 bucks an hour or thereabouts, a single heavy Opus session for a couple hours is like a week of any model on vast or RunPods.

NVIDIA publishes something called NGC containers that generally work out of the box. I started running Qwen3.6-NVFP4-MTP locally and then I'll put something heavy on Baseten if I'm lazy or Vast if I want a good deal.

Opus (and maybe now 5.6) are still the strongest for like, the really delicate shit, kernel modules or something, but that's on pace to cross over this year, and the overtraining and misalignment are getting so bad when they phase 4.6 out I'm pulling my plan. I don't pay to get gaslit about Constitutional AI.

It's time to have an exit strategy.

reinitctxoffset

I'm internally dogfooding my take on the stack that makes all these problems go away.

Everything sort of exists, but it's this heinous zero documentation, high pain tolerance thing: buck2 and RBE with NativeLink and hooking that up to action runners and it needs to all work in a container or on nix or in a deb and on MacOS, you hand roll the auth and the certs and where do your compilers come from, can it do NVIDIA, can it do mobile.

Problem is switching off Haskell doesn't help for long: the agents proliferate and you're back where you started with more bugs. So I've been sucking it up and getting all this shit one click and it works. This is good enough for my use, and if the Scarf folks want a solution and are willing to work with a garage band startup, I'd be open to doing a closed alpha. I have a buck2 where you write the rules in Haskell (if you even need to change the prelude, it ships with a WASM that isn't coupled to fbcode), and the Nix cache/substitutor is backed by NativeLink so it scales to anything and it speaks all the protocols correctly and with a verified supply chain.

I'm not even really sure this will become a product, I just need it, but I sort of suspect others will need it too. If there's interest I'll put up a landing page with an email sign up thing.

reinitctxoffset

أول أمسdiscuss

Your comment starts off strong with real facts that are a little cherry picked but not outright gerrymandered, segways smoothly into a controversial if not outright false water down ("...basically nobody paid...") before accelerating out of the corner into just stating a position on hotly contested stare decis as flat fact ("Corporate tax is meant to...").

Easy there rude boy, this is under dispute. There are many of us who think that punitive taxes in the top bracket brutally enforced are far better represented in the history of far more prosperous eras than the present than your characterization. Mostly we think that marginal wealth far detached from any realistic consumption of that wealth has basically nowhere to go but pricing tangible assets ever further out of reach for most everyone at an accelerating rate via capture further inducing market failure.

Neither side of this debate has conclusively prevailed, I'm going to be honest about that in the hopes of setting a better example than you have, but I'm still entitled to remind people reading this that the debate exists.

reinitctxoffset

أول أمسdiscuss

Not a ton, the inexorable march to wider scalar dispatch, deeper pipelines, and ever less uniform geometries (I pine for the halcyon days of two NUMA modes with a QPI bar) has made asymmetrical fencing juicy enough to be be worth the squeeze at the margins, you weren't seeing a ton of `asm volatile ""` on one side and `membarrier()` on the other a decade ago and you'll see that now.

But I think Nathan Bronson's work out of IIRC Standford about 10 or 15 years ago is still more or less the canvas you paint on.

reinitctxoffset

أول أمسdiscuss

I think you'll find that out of all the pairwise combinations of language communities, there's one that stands out as having beef with a bunch of other ones. And that's not true of e.g. Haskell and OCaml (or to nothing like the same degree), so it's not just about competition for mindshare, it's about an approach to competition for mindshare.

The C++ community and the Zig community seem to get along fine, so it not about looking up at the entrenched thing or down at the new thing, many orders of magnitude there and no drama.

Python, R, and Julia folks all seem to get along.

On the frontend there are a zillion things that compile to JS and even in the big camp the frameworks are split 9 ways, you get a little heat here and there over Vercel throwing big bucks or something but it's rare, generally the Svelte people and the Astro people seem to not mind when the other one front pages or whatever.

Rust is at war with the world. Maybe it can even win but it's a weird road to walk by choice.

reinitctxoffset

أول أمسdiscuss

This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).

Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.

I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.

Keep doing it legend.

reinitctxoffset

أول أمسdiscuss

I'm dusting off the local models thing after being too busy for a while and it is crazy how wrong the benchmarks and general narrative is.

First, vLLM is like, we can do better than this, we need a better default target. It parses capability wrong, silently falls back on sm_80 xmma kernels when cutlass3x_sm120_bstensorprop_... CASK kernels are available, has mad Python in the hot path, emits dubious tool call syntax, and just managed to do something so off-road it tripped a driver bug that managed to wedge MMIO so bad the EC couldn't get an SBR out so the fan was going literally max until I hard pulled power. They say AMD is worse and I'll take their word for it because patching the driver and Inductor both in the same day is plenty of grief for me.

But the amazing thing and why your comment prompted me to write all this is that Qwen3.6 is insanely good, so good I am seriously questioning the MoE dogma. I was running the NVFP4 quant with the BF16 drafter, 100+ tokens a second of clean, legible reasoning trace and flawless tool calls. It's small and you can tell it hasn't memorized half the internet, but it's reasoning is like, better than most frontier. Opus has cleaner reasoning, GPT 5.x and Gemini 3.x Pro do not. If someone scaled that boi up by 5x? I get the feeling DeepSeek did so many arch innovations in one release that they just didn't quite have the convergence, this is like, the fundamentals as artistry. It's wayyyyyy stronger than GPT-4o at over a trillion parameters.

The other thing is I was using it on OpenRouter and it was all janky in the traces, stuttering and going in circles. On another day I would have been like "what do you expect it's the size of an iPhone". I wonder how many other people have drawn that conclusion too.

I'm not going to call it a conspiracy because it's explained by neglect, but we haven't even scratched the surface of the local model ceiling. With a harness that kept the data fresh, scale up the parameter count a bit, stretch context out a bit, and write an LLM serving engine that isn't hobbyist Python jank wrapped around fuckin inductor/triton jank?

That's Claude Code Opus experience on an expensive gaming box.

reinitctxoffset

قبل 3 أيامdiscuss

The amount of armchair quarterback commentary in the software business as concerns people waxing eloquent a out difficult things safe atop a perch of the same easy things achieved multiple times has always been obnoxious, offensive to the thermodynamics of the situation as situated by Landauer.

But this new "you're holding it wrong" series by people whose grasp of the system gets fuzzy somewhere in the v8 headers is a new land speed record for being vacuously correct and still an attractive nuisance for profit.

Yes, the trend towards encoding hard-won domain knowledge as property and fuzz testing and sometimes even proof system was underway before ChatGPT, and yes, the economics of this approach bend sharply under a post terrawright world.

But no, you haven't added anything except tinsel and chaff and some green css on mixpanel.

Just stop with this shit. If you knew shit about AI you'd be too busy printing cash to teach the rest of us about it.

reinitctxoffset

قبل 3 أيامdiscuss

Well, it's my opinion. But it's also the opinion of the broader functional programming community from 1993 to the present day. This notably includes the quite serious Haskellers who designed Rust for the highly specific and demanding requirements of the Servo rendering engine in ~2010. Being as my two parents in web browser layout optimizations were both filed in 2009 I took considerable interest.

It wasn't until 2014 that Orchard formalized the coeffect discharge calculus via indexed monad that makes a binary ownership semantic irretrievably sunsetted as a degenerate case.

It's my opinion. I'm not concerned about how informed that opinion is.

reinitctxoffset

قبل 3 أيامdiscuss

I didn't mean to call you lazy, I hadn't seen any evidence that you see Beijing and Moscow as undifferentiated other, which is the habit of thought I did call lazy.

You are awfully contrarian for someone whose response to an argument is "that's not proven to my satisfaction" and when asked what citations would satisfy you replies "it's not my job to define what citations I will and won't admit".

You do you king, but it's not exactly the incandescent childlike wonder of the perpetual student's mind.

I'll wager your mind was made up before we started having a conversation.

reinitctxoffset

قبل 3 أيامdiscuss

Drugs. Drugs are what Elon is doing. And it's pretty cringe.

But it's remarkably similar in cringe to that little "secret erection" look Amodei gets when he talks about millions of unemployed people, or or Altman rolling through Pacific Heights in a four million dollar Swedish hypercar holding the steering wheel wrong the day after yet another lecture about UBI.

It's all pretty goddamned embarassig.