The Humans Won't Be Called Back(threadbaire.com)
threadbaire.com
The Humans Won't Be Called Back
https://threadbaire.com/blog/posts/the-humans-wont-be-called-back.html
4 comments
> the part thats missing from this whole discussion is that the cost of running frontier models is heavily subsidized right now. companies are making "ai efficiency" decisions based on pricing that doesnt reflect actual compute costs. when inference and r&d prices normalize (and they will) the roi math that justified cutting your engineering team just stops working. but the people are already gone
Lots of people talk about that. It's unclear what the actual cost of inference of SOTA models is, but there are open models with closely trailing scores whose cost on a standard cloud compute system strongly suggests that if these businesses were pure-inference companies who weren't in a constant expensive race for training newer better models then at a minimum Anthropic and OpenAI would be making a profit. (Training is really expensive! And there's no current dynamic that would suggest this stops until the money runs out).
However, even if they're currently losing money on inference, the cost of compute appears to still be trending down. Hard to be sure given the massive demand spike, let alone international geopolitics, but it certainly seems to be. Because of that, even if the estimated cost is off by a factor of 10, we should still expect SOTA models to become cost effective for inference within a few years.
Even more strongly, the models are currently mostly running on general-purpose hardware; if the weights were baked into hardware, they could all be done much more efficiently. You don't even need it to be digital logic, given how few bits get us good results the loss of precision that comes with analogue circuits clearly doesn't matter too much.
Even worse than that, even without all these things, the current behaviour of Claude seems to be like a fresh graduate, who would be worth spending thousands per month to get rather than merely tens or hundreds. Perhaps I got lucky? Certainly my experience of OpenAI's Codex, even on "extra high", is more like a student than a fresh graduate.
> also the nber finding is wild - 80% of firms report zero productivity impact from ai. zero. but layoffs attributed to ai are 12x what they were two years ago. its not ai displacement, its a market correction using ai as cause. the resume.org survey where 60% of hiring managers admitted they played up the ai angle because it sounds better than "we need to cut costs" pretty much says it out loud
Indeed. From what I hear, people claiming layoffs due to AI are basically lying: the layoffs are because of interest rates.
Though "we need to cut costs" would also happen if AI was better than all humans (it isn't, but even if it was this statement is basically eternal under capitalism, it's kinda the specific thing capitalism optimises for).
> the article is right that reversals wont happen. id just add that even if they tried, the knowledge of what "bringing people back" means will have left with the people they cut
Sure. But also, IMO most of the interesting stuff is done by individuals, while the big teams we saw (not just in the zero-interest-rate period but in general) have a hard time producing much of value. Not saying big teams never do stuff right, they do, but it's really hard for them to be organised that well. So cutting a lot of people and leaving only highly motivated nerds may be a good thing.
Lots of people talk about that. It's unclear what the actual cost of inference of SOTA models is, but there are open models with closely trailing scores whose cost on a standard cloud compute system strongly suggests that if these businesses were pure-inference companies who weren't in a constant expensive race for training newer better models then at a minimum Anthropic and OpenAI would be making a profit. (Training is really expensive! And there's no current dynamic that would suggest this stops until the money runs out).
However, even if they're currently losing money on inference, the cost of compute appears to still be trending down. Hard to be sure given the massive demand spike, let alone international geopolitics, but it certainly seems to be. Because of that, even if the estimated cost is off by a factor of 10, we should still expect SOTA models to become cost effective for inference within a few years.
Even more strongly, the models are currently mostly running on general-purpose hardware; if the weights were baked into hardware, they could all be done much more efficiently. You don't even need it to be digital logic, given how few bits get us good results the loss of precision that comes with analogue circuits clearly doesn't matter too much.
Even worse than that, even without all these things, the current behaviour of Claude seems to be like a fresh graduate, who would be worth spending thousands per month to get rather than merely tens or hundreds. Perhaps I got lucky? Certainly my experience of OpenAI's Codex, even on "extra high", is more like a student than a fresh graduate.
> also the nber finding is wild - 80% of firms report zero productivity impact from ai. zero. but layoffs attributed to ai are 12x what they were two years ago. its not ai displacement, its a market correction using ai as cause. the resume.org survey where 60% of hiring managers admitted they played up the ai angle because it sounds better than "we need to cut costs" pretty much says it out loud
Indeed. From what I hear, people claiming layoffs due to AI are basically lying: the layoffs are because of interest rates.
Though "we need to cut costs" would also happen if AI was better than all humans (it isn't, but even if it was this statement is basically eternal under capitalism, it's kinda the specific thing capitalism optimises for).
> the article is right that reversals wont happen. id just add that even if they tried, the knowledge of what "bringing people back" means will have left with the people they cut
Sure. But also, IMO most of the interesting stuff is done by individuals, while the big teams we saw (not just in the zero-interest-rate period but in general) have a hard time producing much of value. Not saying big teams never do stuff right, they do, but it's really hard for them to be organised that well. So cutting a lot of people and leaving only highly motivated nerds may be a good thing.
fair points. i was probably overstating the subsidy argument as if its some trap that snaps shut. compute costs do trend down and if moores law dynamics still roughly apply to accelerators then even a 10x gap closes in a few years. the analog/ASIC weight-baking point is interesting too, hadnt considered that as a near term efficiency path but makes sense given how low the precision floor is
where id still push back is that "inference becomes cheap" and "ai replaces good engineers" are two different claims. inference getting cheap means the floor tier of ai-assisted work becomes very accessible. but the gap between what claude can do as a fresh graduate (generous but i see what you mean) and what a senior with deep system context can do isnt a cost problem its a capability problem. 100x cheaper inference doesnt close that gap, it just means more cheap output faster
on interest rates yeah thats basically my argument stated more precisely. rates went up, cheap capital disappeared, companies that hired assuming perpetual growth had to correct. "ai efficiency" just plays better on an earnings call than "we overexpanded during ZIRP"
your last point about individuals vs big teams is actually a stronger version of the articles argument than the article itself makes. if most valuable work is done by individuals or small groups then the real loss from layoffs isnt headcount, its losing the specific 5-10 people who actually understood the system. the bloat was arguably already dead weight. but the cut doesnt discriminate, it takes both
where id still push back is that "inference becomes cheap" and "ai replaces good engineers" are two different claims. inference getting cheap means the floor tier of ai-assisted work becomes very accessible. but the gap between what claude can do as a fresh graduate (generous but i see what you mean) and what a senior with deep system context can do isnt a cost problem its a capability problem. 100x cheaper inference doesnt close that gap, it just means more cheap output faster
on interest rates yeah thats basically my argument stated more precisely. rates went up, cheap capital disappeared, companies that hired assuming perpetual growth had to correct. "ai efficiency" just plays better on an earnings call than "we overexpanded during ZIRP"
your last point about individuals vs big teams is actually a stronger version of the articles argument than the article itself makes. if most valuable work is done by individuals or small groups then the real loss from layoffs isnt headcount, its losing the specific 5-10 people who actually understood the system. the bloat was arguably already dead weight. but the cut doesnt discriminate, it takes both
> where id still push back is that "inference becomes cheap" and "ai replaces good engineers" are two different claims. inference getting cheap means the floor tier of ai-assisted work becomes very accessible. but the gap between what claude can do as a fresh graduate (generous but i see what you mean) and what a senior with deep system context can do isnt a cost problem its a capability problem. 100x cheaper inference doesnt close that gap, it just means more cheap output faster
Absolutely agreed on all of this. There's problems which you can solve by organising a bunch of juniors, and there's problems they can't do. Even if my experience is typical, which I accept it may not be, Claude is still not a senior developer. That said, I wouldn't phrase it as "with deep system context", the AI are superhuman at context, they're just kinda… weirdly off, even with that context.
For now it sees things in a mirror, dimly lit (if you will excuse the Biblical reference I got via Star Trek).
> the bloat was arguably already dead weight. but the cut doesnt discriminate, it takes both
It can do, sometimes, but I think it doesn't matter much. The competent developers keep developing just in a new place, while the old products which they no longer work on were often good enough long ago.
Absolutely agreed on all of this. There's problems which you can solve by organising a bunch of juniors, and there's problems they can't do. Even if my experience is typical, which I accept it may not be, Claude is still not a senior developer. That said, I wouldn't phrase it as "with deep system context", the AI are superhuman at context, they're just kinda… weirdly off, even with that context.
For now it sees things in a mirror, dimly lit (if you will excuse the Biblical reference I got via Star Trek).
> the bloat was arguably already dead weight. but the cut doesnt discriminate, it takes both
It can do, sometimes, but I think it doesn't matter much. The competent developers keep developing just in a new place, while the old products which they no longer work on were often good enough long ago.
also the nber finding is wild - 80% of firms report zero productivity impact from ai. zero. but layoffs attributed to ai are 12x what they were two years ago. its not ai displacement, its a market correction using ai as cause. the resume.org survey where 60% of hiring managers admitted they played up the ai angle because it sounds better than "we need to cut costs" pretty much says it out loud
the other thing nobody talks about is what happens to the junior-to-senior pipeline and tacit knowledge transfer. its not just that fewer juniors get hired. the ones who do get hired learn in an environment where the hard cognitive work is outsourced to a model. they never build the mental models you need to actually understand systems deeply. shen & tamkin literally measured this - ai assistance impairs conceptual understanding. so you end up with a generation that can prompt but cant debug from first principles and the seniors who couldve mentored them are already out the door
the article is right that reversals wont happen. id just add that even if they tried, the knowledge of what "bringing people back" means will have left with the people they cut