When I was in a large org, it was a common failure that a manager would come in from a smaller firm - and assume that our design practices etc. were meaningless overhead.
Inevitably they'd try to do a "rapid prototype" of a feature that had been tried a hundred times before and declared "too hard/unworkable/uninteresting." as they wouldn't talk to those who had done this before, or look at any past work... the invariable result was a project shut down within 6 months and the manager would be shown the door some time later.
The main takeaway from the above is to listen to people around you when you join an org, and observe the habits of successful leaders. Odds are good that at least within this particular org under these circumstances... those habits are the ones that work.
if super chargers were always within 5 minutes, located around desirable locations to spend an hour, did not charge idle fees, and everyone had 30 minutes of free stuff to do at these locations... then it might just work!
This math is bad and shouldn't be used as a justification. It presumes that every charger is always available, that variable battery conditions and traffic conditions make reaching that charger every time, and that drivers have time to drive to the fast charger and spend 30 minutes there every time their car has low battery.
I drive a new EV in the Boston metro, I do not need to drive most days. Charging without an in-home charger is a massive pain even with Superchargers within 20 minutes drive and a 300 mile range.
A trip to the super charger takes about 1.5 hours assuming its available when I arrive, I can only make the trip when the car needs charging or I am wasting time and energy. The exact time I will need to make this 1.5 hour trip work is variable. It depends on battery conditions, traffic conditions, and what I need to do. I cannot assume that the car will be charged when I need it to be - if I need to depart on Friday evening, or unexpectedly needed to get groceries during the week - I am obliged to spend the 1.5 hours at that moment.
To avoid these problems, I generally need to charge the car when it has ~30-40% and for battery protection I can only charge to 80%. Turning a car rated for 310 miles range into one with ~100-120 miles of usable range and a 1.5 hour weekly maintenance schedule. That range could be further reduced to 50-70 miles in winter conditions. The stalls at charging station charge by the minute, so you can't exactly wonder off.
With a reliable home charger, these problems mostly go away, even a slow 5kw charger should charge the car overnight - and a typical tesla installation delivers 11kw. A renter with no ability to plugin and charge overnight will have a tough time with an EV compared to a gas car.
the counter point is that building or selecting the specialized model may cost as much as the lifetime inference costs of the task with the specialized model.
If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.
Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?
The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
I suspect the concern is that model serving is a stateless “simple” problem.
As yet, no one has identified a reliable moat in inference. If the moat is performance, then prices will collapse. Unlike traditional cloud moats around state, operations, and capex management - I can host a model reliably with less than 30 minutes effort.
If I understand correctly, there are more people employed in manufacturing today than there were a century ago. The hollowing of western manufacturing due to policy choices created a false perception that the sector was in decline globally.
it's a common missconception that engineers spend most of their time producing code based on documented requirements in jira tickets.
I'd believe that a complete automation of this aspect of our industry would only be enough to provide a 10-20% boost in productivity. Still impressive, but within the range of "Our team improved our CI, build times, development process etc."
so all we need is someone to leak a sufficiently large amount of claude generations onto the open and private web for all other LLMs to mimic the same marking style?
wouldn't this happen due to the massive amounts of spam/slop being released?
Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc.
Whether such a model exists or not is a different question.
The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
There is some signal that this is possible through both hardware innovation and training/data improvements.
Cloud models have their own constraints - I can’t have opus4.8 spend 4 hours on a deep research question I had in the shower without spending money. I can’t do real time video game upscaling and graphics work in the cloud period.
A laptop is about an order of magnitude cheaper than a cloud server thanks to economies of scale, uptime requirements, and other factors.
The s curve won’t inflect until it becomes difficult to allocate additional resources due to economic limitations. There is no sign that training a model on 10x the compute won’t lead to at least an equivalent improvement as the last order of magnitude increase.
If we define the Pareto frontier’s input in terms of a magic “compute equivalent unit”. We get a free order of magnitude from nvidia hardware improvements every 2-3 years. We get another order of magnitude from capital expenditure every 6-12 months. Kernel improvements to the models themselves likely yield an order of magnitude gain at some periodicity.
I don't think this is really true, there are plenty of engineers/managers who rotate through major tech companies. Many Meta folks will head off to new companies which would pay at similar levels.
This is true if you take the ai market as equal to the market for labor discounted to 5-10% penetration.
It’s not a totally unreasonable assertion, it’s the implication of the assertion that we are uncomfortable with. There is no reason for the models to stop their improvements in the near future.
The abstraction of capital and money get a bit funny when wealth is sufficiently concentrated. If there is a monopsomy (one buyer), then they can largely dictate the price of anything. If they also control violent coercion via a captured state or other means, then they can compel production at that price point.
The idea of capitalism only really makes sense when wealth is reasonably distributed such that there is still reasonable competition in both the marketplace and control of the state.
This just looks like a capex problem. There is no evidence that Anthropic has secret sauce above and beyond access to capital. If there is secret sauce, it's unclear that it changes the required amount of capital by all that much.
China will spend all of the money required to catch up, Google and OpenAI will both spend money to catch up as well. NVidia and others will not allow a frontier lab to become the AI bottleneck.
Inevitably they'd try to do a "rapid prototype" of a feature that had been tried a hundred times before and declared "too hard/unworkable/uninteresting." as they wouldn't talk to those who had done this before, or look at any past work... the invariable result was a project shut down within 6 months and the manager would be shown the door some time later.
The main takeaway from the above is to listen to people around you when you join an org, and observe the habits of successful leaders. Odds are good that at least within this particular org under these circumstances... those habits are the ones that work.