Agreed that seems like a potential conflict that should be scrutinized, but it highlights a real problem in health research. Who else would fund a 40,000 participant 15 year study about egg consumption and Alzheimer’s? How many studies are not conducted because no one is interested in funding them?
I think the value right now is to focus less on external orchestration if at all. trust the (current best) model to do it better than anything you bolt on to the harness. focus your energy on providing clearer specs. I think the optimal spec is a disambiguated (through liberal use of the AskUserQuestion tool) 1 intent, 2, input/output contracts 3 constraints and 4 preconditions. focus on that and get out of the models way. I think of it like this, imagine a person who was not as smart as you was trying to tell you how to do a task. would you want more verbosity and step by step instructions or would you want them to just cut to the chase (ie, what are you trying to do, what are the obstacles, I'll let you know if I have questions).
also let the model verify itself. don't give it an objective that is vague, give it clear exit criterias for goals and let it loop until it gets there
so much of the orchestration scaffolding seems like massive technical debt
oddly, I do the opposite of a lot of conventional advice when it comes to models. I use no memory, I think there is something similar to context rot when everything is stored. I like creating markdown files as memory that the model can grep if needed. I also havent found a real use for hooks yet, I have tried but they always seem to get in the way. skills on the other hand are very undervalued. they are so much more powerful than many realize. I used to think agents were where the power was. I think its actually skills. agents are really for context preservation. skills are what increase capabilities
I'm not even talking about quantity of items in memory, I mean dilution of intent. I really love a model with a clean slate and only the items it needs. I fear the memory guides the model in areas that might not be what I want with the current prompt
progressive disclosure is a big one. you can make context available but it is only loaded when needed. like lazy loading for prompt engineering. skills are to be used to instruct the model how to do something specific that is not in its training data. like how to access my proprietary system, how to interface with a custom program. you can embed templates in skills, you can embed code that executes in skills and only the output is loaded into context. skills expand capabilities, agents constrain context
(constraining context is a very good thing btw, don't mean to infer that agents are somehow inferior to skills)
I don't see any ethical connection between adding canary tokens to your output to catch people breaking your accepted ToS through ongoing distillation and stealing your PII off of your machine. How are legitimate users in US or China possibly harmed by Anthropic silently changing the apostrophe in Today's or the date separator from - to /?
Respectfully, the story you referenced is about how MSG compiled "Facial Recognition Activists.docx," collecting its critics' tweets and comments into an internally accessible file.
“On one end £9 of labour cost for a plate of asparagus seems deeply inefficient and unrealistic”
Every time I’ve had this thought that I could recreate a dish and spend a lot less, I end up paying more for the same dish, with lower quality and I still have to do the dishes. It might not be as unrealistic as it seems if you aggregate the wages of all staff involved and divide it by the number of plates they serve over a night. I’m not a cook, but if someone offered me $9 to prepare them a comparable asparagus dish, it doesn’t sound that lucrative.
If this interview isn’t important enough to assign a human to the task, it sends a message about what you should expect as an employee. I’d expect a larger percentage of these AI interviewers will be vetting AI interviewees.
The reason there’s no good orchestration layer above the base model is that the useful part of orchestration moved into the model. what’s left to bolt on top is negative value when token consumption, latency and usability are factored in.
I’ve spent a great deal of time trying to beat plain Claude Code and Codex with a variety of planner/critic/decomposition orchestration setups. Every one produces less or same quality, 2x+ tokens, more latency and a second system to debug. External orchestration seems like I’m just creating technical debt and the actual value is focusing exclusively on providing clearer specifications (ie statements of intent, constraints and input/output contracts) to the model and just letting the leading frontier model do its thing with reasoning and effort maxed out.
Ironically the real orchestration benefit is on the human side refining intent to model instead of orchestrator to agent, invoking AskUserQuestion and other tools for the model to more deeply disambiguate my requests.
“But it appears 1 or more organizations have successfully jail-broken Fable 5”
This is hardly true or it’s true of all frontier models and this was only magnified by Fables capabilities. It’s that you could hand Fable 5 vulnerable code, ask it to fix it, return patch plus test cases proving the fix and exploit relevant detail falls out as a byproduct of legitimate secure code review work.
I challenge anyone to provide a fix for this “exploit” without compromising Fable’s ability to patch unsecure code.
By AI fluency I mean basically knowing how to drive the tools, an intuition for what the tools can and can't do, when to use AI vs doing it yourself, plus detecting when output is wrong, knowing what to verify, etc.