cgearhart

1,964 karmajoined vor 16 Jahren

[email protected]

comments

vor 3 Tagendiscuss

We used to use this, but it was a broader conversation around tradeoffs to meet different constraints. If the expected array is small, then sort + index is probably fine. If it’s big (bigger than main memory?) and latency is the most important then maybe you want median-of-medians. If it’s a stream and you want to keep memory fixed then you might want a sketching algorithm. If I suggest that we can bound the error of the median estimate with constant additional space and the same complexity, would you believe me? (Just track the mean and standard deviation.)

Honestly, when I ran this interview I didn’t care much about the specifics of what you memorized beforehand. I care if you can read and write code a bit. I care more whether we can have a productive conversation. If you learn something new from me or the problem, how does that look and feel? If I make a mistake, how do you react? Are we able to communicate technical ideas to each other? Are we able to productively work through conflict?

We’re not computing many medians day-to-day, but we’re doing all those other things constantly.

vor 5 Tagendiscuss

My read has been that a lot of leaders were trying to drive “being early” as the catalyst for future success. At the complexity scale of big orgs you’re mostly fiddling with the incentives that the system self-aligns toward. Firing a bunch of people does create an incentive to use AI, if you think it’ll help.

The more pernicious effect I’ve been seeing is that we’re living in the golden age of LLMs, but eventually that’ll fade. Tokens are subsidized and cheap, model capabilities leap forward regularly, and there’s competition driving it all. But even now there’s stories about frontier models suddenly becoming less capable, or providers switching to usage-based billing, and new model releases feel a bit more sluggish and less dramatic. (Fable/Mythos notwithstanding.)

Eventually the models are going to settle into a rut of being just “good enough” to earn a living rather than all this hoopla. A lot of people will be re-hired. And we’ll do it all again for the next wave.

vor 5 Tagendiscuss

Yes, that’s what I think at this point. There is no effect of the study group except as a support group. (That’s all it was for me when I was a student and joined the self-organized study group.)

vor 6 Tagendiscuss

I used to TA a graduate level CS math class at Georgia Tech. We regularly saw that the students who self-organized study groups did dramatically better in the course than average. One semester they told us to put everyone in study groups to see if it helped. The effect disappeared. Turns out that it was the self-selection of the most engaged students into a small group that mattered, not the study group itself.

letzten Monatdiscuss

I’m starting to realize that LLMs are really good at building low-stakes projects. Your questions mostly presume that the stakes are higher. The software will last a long time; the requirements will evolve; we can’t tolerate mistakes; etc.

The trick to getting good at using LLMs for software is to learn how to make _all_ projects low-stakes.

letzten Monatdiscuss

I just tested this myself. I wrote “flip the reduce white point toggle accessibility option in the settings app” and it worked perfectly. Run once to set it and run again to disable it.

vor 2 Monatendiscuss

Spreading out the refusal encoding shouldn’t be effective as a countermeasure. Even if it were smeared across the vector space, as long as it’s in a subspace that doesn’t span the entire domain then you should be able to either null out the entire subspace spanned by the refusals or run some kind of clustering on the generated samples to identify the dominant directions and nullify all of them. I think an effective defense would either need to spread them to span the entire domain—basically “encrypting” the refusal so it can hide anywhere, or you’d need a very large number of independent refusal circuits in the model so that simple hacks in the vectors themselves don’t matter, or maybe you could make other circuits depend on proper functioning of the refusal circuits… hmmm… is that along the lines of what you’re saying they’ve done already? (Any references or links to modern techniques?)

vor 3 Monatendiscuss

A much earlier major win for deep learning was AlexNet for image recognition in 2012. It dominated the competition and within a couple years it was effectively the only way to do image tasks. I think it was Jeremy Howard who wrote a paper around 2017 wondering when we’d get a transfer learning approach that worked as well for NLP as convnets did for images. The attention paper that year didn’t immediately dominate. The hardware wasn’t good enough and there wasn’t consensus on belief that scale would solve everything. It took like five more years before GPT3 took off and started this current wave.

I also think you might be discounting exactly how much compute is used to train these monsters. A single 1ghz processor would take about 100,000,000 years to train something in this class. Even with on the order of 25k GPUs training GPT3 size models takes a couple months. The anemic RAM on GPUs a decade ago (I think we had k80 GPUs with 12GB vs 100’s of GBs on H100/H200 today) and it was actually completely impossible to train a large transformer model prior to the early 2020s.

I’m even reminded how much gamers complained in the late 2010s about GPU prices skyrocketing because of ML use.

vor 3 Monatendiscuss

I agree, it would be nice if we could prioritize basic human needs rather than treating them like burdens caused by bad luck or poor choices.

vor 4 Monatendiscuss

Slightly unrelated to this story, but I’m curious if anyone has good resources for learning FreeCAD. I have quite a lot of experience with SolidWorks, AutoCAD, OnShape, and similar software, but FreeCAD has always been hard for me to pick up.

vor 4 Monatendiscuss

Eh. I think my point is that the OP is presented as a “how to” (literally: “how to do important research”) and then it immediately dodges the question by saying “have good taste”. That does not help anyone do important research or improve the quality of the research they do; it’s a cop out.

If I wrote about “how to paint great art” or “how to cook great meals” or “how to build great things” then it would be silly to say “have good taste”—even if that’s part of the answer. It won’t help anyone else to improve in any of those endeavors.

vor 4 Monatendiscuss

That seems even less actionable, and somewhat misaligned with the OP article. “Taste” implies an ability to distinguish between a good example and a bad one. If it’s only recognizable in retrospect then it’s just another name for survivorship bias.

vor 4 Monatendiscuss

I often find this kind of advice too vague to really be useful. “Have taste” in the problems you work on isn’t very actionable. (Unless perhaps you list examples of good and bad taste.)

I’ll admit that I may just be immature at research as almost all my experience has either been attempting to replicate research or to put it into practice in production systems.

vor 5 Monatendiscuss

Any notes on the problems with MLX caching? I’ve experimented with local models on my MacBook and there’s usually a good speedup from MLX, but I wasn’t aware there’s an issue with prompt caching. Is it from MLX itself or LMstudio/mlx-lm/etc?

vor 6 Monatendiscuss

I think this is the actual “bitter lesson”—the scalable solution (letting LLMs bang against the problem nonstop) will eventually far outperform human effort. There will come a point—whether sooner or later—where this’ll be the expected norm for handling such problems. I think the only question is whether there is any distinction between problems like this (clearly defined with a verifiable outcome) vs the space of all interesting computer programs. (At the moment I think there’s space between them. TBD.)

vor 7 Monatendiscuss

So…great for prototyping (where velocity rules) but somewhere between mixed to negative for critical projects. Seems like this just puts some mildly quantitative numbers behind the consensus & trends I see emerging.

vor 7 Monatendiscuss

“Better than me” != “good”

I know approximately nothing about approximately everything. Claude seems pretty good at those things. But in every case I’ve used Claude Code for something I do know about it’s been unsatisfactory as a solo operator. It’s not useless, but it is basically useless for anything serious unless you’re very actively guiding it.

I think it has a lot of potential value and will become more useful over time, but it’ll be most useful when we can confidently understand the limitations.

vor 7 Monatendiscuss

I love the way you phrased this.

vor 7 Monatendiscuss

Best of luck! I hope it’s as amazing for you as it was for me.

vor 7 Monatendiscuss

I had been working in civil service for the US Navy for about 10 years in operations research & systems engineering. It was very hard to break out of that role to any private industry—especially for the ML roles I wanted, which I think was partially because my undergrad degree was MechEng.

OMSCS allowed me to add MSCS to my resume, with additional networking and work experience details as a TA for the algorithms and Computational Photography courses. Suddenly I started getting a lot more calls back. About 6 months after graduation I had moved to the SFBay (to work for Udacity) and within 2 years I was an ML engineer at Apple where I remain today. I don’t think any of that would’ve happened without OMSCS.