> I don't think most software is like solving a math problem or series of math problems.
I agree with you when talking about high level software design. As you say it ultimately boils down to building something people will pay for, which is a fuzzy correctness function that is hard to measure within an agentic sandbox.
But unlike other professions, there are a lot of sub-problems within software development that are able to be fully specified and tested via text generation. And I think the developers of AI overestimate how many such problems exist for other professions. What I’m saying is most other professions tend to be “fuzzy all the way down”… which incidentally is why they select for people with fuzzier skillsets.
It seems like a solid set of criteria for how easily a task can be automated by AI agents is:
- extent to which correctness of solution be easily specified and checked
- extent to which new potential solutions can be implemented as text
- extent to which prior art exists online
This basically maps to software engineering and math. I think a fair bit of AI hype comes from the fact that the very architects of AI are the people whose jobs are most easily automated by AI. They think, “if my job receives this much of a boost from AI, surely every job will be the same”. Ironically it couldn’t be further from the truth… and likewise the predictions of widespread labor obsolescence
Does anyone really believe he’s doing all of this just to sell out and live on a beach somewhere? This dude sleeps at his own factories. He literally works like 24/7 and has no personal life. If this was all a cash grab he’s had dozens of opportunities to cut and run with well beyond F U money. If you wanted to scam people there are a lot easier ways to do it than repeatedly founding revolutionary technology companies.
I’m not denying that his companies are awash in zany financials, but I don’t think that’s ever been the point
As much as it’s tempting to read some kind of ulterior motive into this, I think the most reasonable explanation is that AWS, as perhaps the single biggest point of failure in the backbone of US IT infrastructure, has legitimate concerns about its ability to fend off attacks from bad actors armed with the most advanced models.
> The first requirement is that the computer program has a body (either physical or virtual) and sense organs
Ok, deploy a local model on a lightweight edge compute device and strap it to a chassis with wheels, and attach a cheap webcam
> Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can
Give the robot appendages that enable it to plug itself into a standard wall outlet, guided by a vision model plugged into its webcam. As long as it can feed itself, it can survive long enough.
> Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse.
I think if you fed frames from the webcam into a local VLM every 5s you’d be able to assess a situation and respond with simple actions (turn, advance, retreat).
> After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees.
Social dynamics could be implemented in many ways, maybe by transmitting tokens over RF? Idk. Then you have a scanner that picks them up, feeds them into some LLM frontend and decides whether to add them to a global context file that guides the VLM action-taker. A new action could be to broadcast a token message. Tool-making would have to be code-based. Physical tools are hard. Still unsolved.
> At that point I would want to see people successfully teaching such embodied agents how to communicate their desires
This part is relatively straightforward except for the “via nonlinguistic modality”.
Anyway. These are all engineering problems. Personally I would demand to see the AI reproduce its body under its own power and volition. That’s a pretty neat trick we’ve got going for us.
> Altman’s realization was born partially out of an experiment of his own. He tried delegating his Slack and email responses to AI, then began responding to some again manually.
There may be additional major leaps forward, and there may not. I kind of struggle to imagine what the next step actually is. Certainly there will be improvements in performance (speed) and cost. But at a point you reach a barrier where the limiting factor is the specificity of the human prompt and our ability to manage all the code we’re generating.
Somewhat oversimplifying; writing software and building apps was a bottleneck - now it is not. What is the next bottleneck that LLMs can solve? Is there one? And is there enough publicly available data to solve it repeatably at scale? Or did we just automate stack overflow searches and now we’re stuck again?
Or is the endgame of this innovation cycle the complete removal of interaction with machines through code? Will we simply interact with machine coworkers purely through natural language? Can an LLM make PowerPoint slides and run a meeting? So far not seeing much progress on that.
Our shop is forced to use Copilot on gov cloud, and it’s so useless I usually stick to manually coding. Its syntax is messy, it randomly combines lines together, flips order, or drops a couple tokens worth of output in the middle of a line, and for some reason it consistently drops the last line of every code block. I assume we’re getting a few versions back of GPT under the hood. But it does make me appreciate how the models of the past year or so crossed the threshold from interesting to truly productivity-enhancing.
Between Copilot, Claude, and Gemini, I still actually prefer Gemini. I do a lot of scientific writing in addition to coding and Gemini is the only model I can trust to “just be right”. This trust then transfers over to its code output.
kind of sobering to realize that whether your job can be profitably automated away comes down to what $/token some hyperscale AI provider can deliver… I suppose it’s nice that this article highlights some upward pressure on that number.
If I’m understanding this correctly, it starts by raising this question, then argues that it’s kind of a cheap and meaningless punch-down.
I’m not sure I completely agree.. I think these types of questions are more a response to the level of hype around LLMs and less about knocking people down. You see a lot of people excited about the personal productivity app they vibe-coded (which IMO is totally legit - it’s cool that run-of-the-mill apps that used to require a professional developer are now available more on demand), and yet it’s hard to think of a new piece of high-impact traditional software that has come out since the release of ChatGPT.
But it’s also hard to think of the most recent piece of important traditional software that came out… at all. I couldn’t even name the most recent Photoshop-like release. Ableton / Fruityloops? Tableau? Big pro-sumer apps kind of plateaued in the early 00s.
LLMs have made it easier to develop software, but at the same time they’ve also raised the bar of what’s worth writing software to do. Many things that used to be apps are now just prompts. Maybe ChatGPT was the next Photoshop - it turned writing basic apps from a profession into a hobby.
Anyway. Good post - definitely not written by an LLM, and that’s a good thing.
> Teams is where most enterprise work happens: decisions get made, customers get answered, and projects move forward there.
… and Teams is where it will stay! Microsoft’s piss-poor integrability across its productivity suite is the single biggest reason agentic AI will fail to deliver its promised productivity benefits. You probably could build an agent to do a lot of people’s jobs, provided it had unfettered access to Teams, Outlook, Word, Excel, and PPT. But I can’t think of an organization who would want to grant that access today, and even if they wanted to, the solution here is to expose a self-managed HTTP endpoint for all your Teams traffic? Seriously?
No, you’d remark that your house has appreciated in value over the past 20 years. But you wouldn’t have realized any of that gain until you sold the house - the point being that the realization is the actual taxable event, which is why it matters from the pedantic technical accounting POV. The fact that you turned around and bought another house just means you’re doing something new with your realized gains. Now you have a new cost basis. Maybe that’s what you’re saying with “unrealized gain” though.
It’s funny, I hear the exact same phrasing used when justifying Tesla’s valuation. “It only makes sense if…” … if you ignore what the actual, physical business does today, and picture it doing something entirely different, beyond its current capabilities (robotaxis, androids, etc)
The difference with this pie-in-the-sky ambition (Mars Colony) is that I don’t even understand how it would be profitable if achieved. What do you get from a Mars colony? What on earth (no pun intended) could you extract from it that would command that amount of value? This isn’t like colonization of the americas, where there was a trove of readily available natural resources to extract and sell back to the mainland markets - nothing is going to get shipped back from Mars any time soon. A Mars colony could only be supported through significant public investment - so is the valuation justified via the expectation that SpaceX will be the primary vehicle for public investment in Mars exploration, or through the centuries-long payback period of founding a self-sustaining civilization? Or both?
Came here to quote the same sentence, but say the exact opposite - it seems to me that today’s LLMs are progressing far faster on the “thinking” front than the “doing”.
I suppose it depends on your definition of “doing” - if it’s “writing code”, then sure. But there’s a whole world of actual, physical “doing” that AI is nowhere close to matching humans at, and it’s much easier for me to envision a world where AI replaces the management / “thinking” layer of society than the physical labor. Which is scary, because it’s the opposite of his (and I would assume most people’s) ideal.
I agree with you when talking about high level software design. As you say it ultimately boils down to building something people will pay for, which is a fuzzy correctness function that is hard to measure within an agentic sandbox.
But unlike other professions, there are a lot of sub-problems within software development that are able to be fully specified and tested via text generation. And I think the developers of AI overestimate how many such problems exist for other professions. What I’m saying is most other professions tend to be “fuzzy all the way down”… which incidentally is why they select for people with fuzzier skillsets.