minimaltom

เมื่อวานซืนdiscuss

I agree you probably don't need top-dollar bay-area engineers for this, but hardcore outsourcing to a LCOL probably isnt going to work either due to novelty and generally being setup to do the more rote thing (generalizing a ton here obvi). This feels like something in the middle.

minimaltom

5 วันที่ผ่านมาdiscuss

This, taken in combination with the SAE paper, the golden-gate claude paper, the feelings / introspection paper, and note in the fable system card (that they are silently nerfing responses about activation shaping), is basically confirmation to me that they have a new technique they they are using during training (along the vibe space of these mechinterp papers), and its probably some kind of representation learning akin to the core ideas of JEPA.

(Nb: not an expert / in the labs, just opining)

minimaltom

เดือนที่แล้วdiscuss

If you know whats going on, could you link to em?

Somewhat similar to you, I also got the sense that this was a pointed response/positioning to some controversy. Not because I have any clue whats going on (I dont), but just cuz it pattern-matched with the style/tone of a PR statement.

minimaltom

เดือนที่แล้วdiscuss

Frontier labs have their own variants of MLA and certainly their own balance/scaling-laws for things like MoE vs FC vs Attn. MoE scales really well for inference with horizontal scaling + batching, which these guys luv.

On the architectures side, I'm a lot more interesting in attention residuals than anything else, one of those things that seems obvious in hindsight and Kimi have proven it at scale.

minimaltom

เดือนที่แล้วdiscuss

Yes. A tax on the ultra-wealthy, rather than a measure aimed at increasing housing.

Its very roundabout as NYC can only make taxes for NYC, but the net aim is to increase the effective tax rate for the ultra-wealthy, using secondary property as a proxy for that.

Edit: AND WE (I) LIKE THIS because progressive taxation is the core play of fixing income/wealth inequality

minimaltom

2 เดือนที่ผ่านมาdiscuss

Theres no gotchas in the quoted text, but curious if you got the impression from your emails that any of it was.. AI generated?

The camber, affirmation, word choice, triplet phrase... leaves me wondering. But without a smoking gun its hard to know if a model call was fired.

minimaltom

2 เดือนที่ผ่านมาdiscuss

In this case you can say "the contract we reviewed was poisoned via technical means to show different words, depending on how the file was read". Perhaps if pushed you can say you loaded it into GCAI/Harvey/Legora and read/reviewed it there.

There is no parallel construction where this wasn't deliberate & malicious, so it seems really high risk given the judge would rip you a new one if discovered.

You can't rely on the defense that the other party didnt read it, if you made it show different words depending on how it was loaded.

minimaltom

2 เดือนที่ผ่านมาdiscuss

It seems like the main attack scenario for this + legal AI would be during discovery: if opposing counsel gave you a poisoned PDF, and you threw it into one of these products to help you sift through it and got bad answers.

However, wouldnt this be a rather risky move? Courts authorized the discovery, so I imagine the judge might loose their marbles and throw the hammer at them if this came to light.

minimaltom

2 เดือนที่ผ่านมาdiscuss

This is a great writeup! thank you!!

Reminds when i did noogler training back in the day and one of the talks described a cascading failure at a datacenter, starting with a cat which was too curious near a power conditioner, and briefly conducted

minimaltom

2 เดือนที่ผ่านมาdiscuss

They absolutely have backups, I presume they were ineffective or also down for _reasons_.

minimaltom

2 เดือนที่ผ่านมาdiscuss

Worth adding also that you can only use these vectors to corrupt the page cache for files reachable in your mount namespace.

Usually with containers, almost nothing is shared with the host namespaces (tho likely shared with other container namespaces, hopefully none of those are --priv).

minimaltom

2 เดือนที่ผ่านมาdiscuss

Nah, even insane token costs don't come close to the costs of labor.

Most likely this is just 'AI-washing' - dressing a layoff for economic reasons (such as propping up their shrinking margins) as something more palatable to investors (AI).

minimaltom

2 เดือนที่ผ่านมาdiscuss

Between this, the emotions paper, golden gate claude etc, it doesn't seem like such a stretch that Anthropic are doing some kind of activation steering as part of training (and its part of their lead)

minimaltom

2 เดือนที่ผ่านมาdiscuss

Theres been multiple papers discussing how only updating parameters that have high agreement in update direction leads to less overfitting and better generalization. Lemme see if I can find em.

https://arxiv.org/abs/2411.16085 - set updates to 0 where theres disagreement in the sign of the parameter update - got accepted!

https://arxiv.org/pdf/2412.18052 - discard gradient updates from batches/minibatches that disagree where disagree means cosine distance threshold (they solved for 0.97 or something being optimal)

minimaltom

2 เดือนที่ผ่านมาdiscuss

This attack class lets you escalate from any user to UID 0. Not running as root won't save you, in fact, this attack is for those processes not running as root.

However, if you are in a user namespace where UID 0 doesn't map to system-wide capabilities, and you dont share page cache for the setuid binaries on the system, this attack doesn't lead to LPE.

minimaltom

2 เดือนที่ผ่านมาdiscuss

> That is, if the batch signal on a parameter exceeds its leave-one-out noise, update it; if not, skip it. This is a one-line change to Adam that accelerates grokking by 5x, suppresses memorization in PINNs, and improves DPO fine-tuning, eliminating the need for validation sets entirely.

Does anyone understand the formula they expressed above this sentence? is this just the classic "skip updating parameters with high gradient/loss variance in multiple batches/samples" ?

minimaltom

2 เดือนที่ผ่านมาdiscuss

my guess is they want to do AI/O as part of their event loop explicitly, and blocking a thread in a syscall waiting for an IOP (ala std::fs) isn't the vibe.

minimaltom

3 เดือนที่ผ่านมาdiscuss

high surface energy?

mahm/father I yearn for the spicy surface

minimaltom

3 เดือนที่ผ่านมาdiscuss

Unless "with experience in the field" == academia, disagree. In particular I remember the early discourse & hype around Wireguard, it was discussed as if perfection was an achieved outcome.

minimaltom

3 เดือนที่ผ่านมาdiscuss

The archive-handling code was in lean-zip, it just seems the verifiers forgot to write proofs for it (still a bug).

Thats not the main finding of the article however. The main bug found was actually in the lean runtime, affecting all proofs using scalar arrays where the size of the array is not bounded.

comments