I'm glad there are more attempts at solving model routing, as costs (at API rates) has really become an issue. Some feedback:
1. Reiterate the cache issue from other comments already here. there is a lot of optimisation in harnesses around caching and a proxy model blows that up
2. Coding agents are model aware - they already route code discovery to mini / flash models, planning to heavy models, workflow design to ultra, implementation to mid / high etc. They know when they're exploring, planning, implementing, reviewing etc. and which model class to select and when it fails.
With a proxy you're breaking this control loop and feedback. It doesn't know, for ex. that it just attempted with deepseek v4 and it failed, lets try Opus?
3. How are you going to RL improvements and prevent the router becoming stale? You only have access to your own internal prompts and ~thousands of samples.
This is RL'd on one orgs codebase. There are going to be a lot of prompts you haven't seen before and have no insight to on how to route correctly, and you have no insight into users HF to improve your own model. Orgs aren't going to share their traces with you, so you need other sources to train on and improve
There are also new model releases every week that you need to keep up with - whats the story going to be here
4. Publish evals by running terminalbench / deepswe bench. Show us the performance / cost / time chart vs the other agent and model sets. If you can show gains there, you have a very simple value prop to sell where you can charge for a % of the saved costs
This is devastating. Om was the godfather of early tech blogging and lifted up so many people around him. He was kind, caring and compassionate.
When I first started blogging around 25 years ago, he would have been amongst the first 10 readers. He linked to me, emailed me privately with feedback, praised posts and would call bullshit when he saw it.
He was never competitive with other blogs or bloggers and was never tied up in drama. He was very often a mediator in behind the scenes conflicts and was obsessed with truth over getting the scoop.
He loved tech and startups and most of all loved seeing other succeed and didn't have a gram of resentment within himself.
Everybody from that post-dotcom crash era of tech owes Om a large debt of gratitude. He will be missed. RIP Om.
> "a well written agents.md is very good for the agent"
while even a mildly bad agents.md can be _very_ bad for the agent. they rot very quickly which is why human curation is essential.
same with memory - a lot of the self-learning tools that are becoming popular now degrade agents over time - which is why you end up being able to run an eval with no context and it performs better
> but that's why agentic coding can still be considered a "skill".
yes - far too many cases of throwing a kitchen sink of prompts, skills, tools etc. thinking the llm will sort it out. you need to constantly prune, eval, tweak, observe, update etc. in a loop
these old network security techniques don't really work anymore. the common bots are at known IP ranges, the problem bots are all on datacenter + residential proxies.
I believe the urgent deprecation timeline here may be related to ai labs using offline licensed Office in agents as part of workflows and Office integration. Microsoft wants _each_ agent instance to be a separate license[0]
There was always a probability that Microsoft were going to funnel offline users into O365 at some point - but I imagined that to take place over months / years not weeks and days.
Buying a single license for thousands of agents may have expedited that. It has resulted in non-Microsoft labs having better ai integration into their products than Microsoft.
edit: just read the detail of the note - so this is a cert expiry as part of Apple dist that is being warned about ~2 months before it happens. Standalone on Mac has a term limit.
The syntax is the easier part - most programming tasks require the reasoning and understanding of a large world model to solve problems.
Fine tuning a 'lean and smart' model works really well for discrete, repeatable high volume tasks like support ticket triage, lead classification, content filtering, labelling, generating content with a voice, etc.
Inefficient token burn by throwing large models at everything is definitely a problem - it's like hiring Phd's to answer the phone or to wash dishes.
the CVE + patch data has been built into models for a few generations now. I actually thought the bug bounty companies were well positioned here, but they've been overtaken.
There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this:
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity.
for anybody who has applied opus, codex or oss models for vuln scanning - the true positive rate and discovery volume are a clear step change[0]. The ~50 partners in Glasswing have largely all previously run harnesses with other models and many of them have come out and said - essentially - "ye, wow"
Question now is what a second and third phases of access looks like - deciding which class of systems to secure. Routers, firewalls, SaaS, ERP systems, factory controllers, SCADA systems, zero-trust VPN gateways, telecoms gear and networks, medical devices - there's just so much to do
This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
That may suit Anthropic as private models can't be distilled. There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
I don't see how Chinese companies are given access soon, or ever. We're likely going to see a world soon of CISA mandated audits, and where to buy a mythos-proof VPN gateway or home router - you'll have to buy American[1].
Anthropic this quarter will have revenue of $10.9B, up from $4.8B last quarter[0]. They're paying SpaceX $1.25B per month for compute[1] - which is more than what SpaceX earn on space. SpaceX spent about $30-40B in capex on Colossus 1 & 2.
This is all real revenue, real spend, real usage.
Hetzner just aren't at this scale. Not even close. If they wanted to get into this business - first, they're late. Second, it's at a scale of ~10x of their total lifetime datacenter buildout. Third, they'd need to change their business to being one that is debt fronted.
xAI have proven out that being able to deploy compute is a very viable business (and difficult to pull off)
At some point AI cynicism clashes with reality, it must be exhausting maintaining it.
This just incentivizes market for bio-mules, which already exists with world[0] - where prices stay low because it was rolled out to low-income countries.
Then there's the platform game theory. If you adopt you add friction which reduces signups, and there will always be a competitor who would risk the 10x fraud increase in order to capture 100x the market. Railway has seen hyper-growth because it's so easy to run from, and is recommended by, coding agents[1].
The solutions are here already just not well implemented or understood - probabilistic fraud detection, resource limits, service and automation limits, standard gov identity verification as a signal, enterprise sales channels with human relationships, etc.
There are tradeoffs with each platform choice that just aren't well understood. Most users shop on price and DX and don't see the abuse infra or problem until it hits them.
Google and GCP have a problem where they completely cook users who get flagged in their automated fraud net (this isn't news - or shouldn't be)
email nik at nikcub.me
@dir on twitter / x