Just prompt it that way then, I can get Claude to plow through features for 12h with the correct prompt and setup, no need to destroy the tools that are useful when you want the opposite.
Yes I'm also using it for coding: I often make the agent use WebSearch in the research phase when deciding on a stack or a library or research best/modern practices to do achieve something. As for images I find it super useful to be able to paste snipped screenshots to show the agent when something is wrong in a UI/frontend or just something I can't copy paste easily.
You seem to have tried a few things, if you don't mind I have a few questions as someone currently on Claude Code but would prefer to not lock myself in a commercial ecosystem (and their pricing change regarding headless usage is annoying me):
- how do/would you add the WebSearch tool to your harness? pay for a separate service or does deepseek offer something with their subscriptions?
- do pi/opencode support pasting images in prompts?
- how do you handle reading images? deepseek is not multi modal IIRC? do you pay for another model and route to it?
Any of these missing would really annoy me in day to day use...
Tangential question for Claude Code subscribers, mid June `claude -p` will move to api pricing (with some "SDK credits" before it kicks in), so headless usage will become 20-30 times more expensive, and all these high level orchestrator tools/workflows depend on it. What the next move for you? How does the OpenAI subscriptions compare? Similar limitations?
I suppose folks here already know this but it deserves a mention: subscription pricing is 10-20x cheaper than API pricing at Anthropic for example and it will be a far better experience (better models, faster responses, as much parallelism as you want, etc) so if it works for you there's no economic argument to buy a machine for inference at the moment.
I think we should separate the private AI discussion from the local AI discussion.
The pragmatic choice to run big LLMs is one/several big servers online, but that doesn't mean private companies should be the only ones to run them.
A self hosted inference solution that offer good tenant isolation guarantees (ideally zero trust) and is easy enough to deploy and maintain (think Plex for AI) would be my choice for privacy. Now to be honest I have done zero research about this and have zero idea how feasible that is, maybe it already exists and there's some discord servers I should join?
Edit: I don't need to mention it here but what's incredible is that open models are in the ballpark of the best commercial models so supposedly, the hardest part by far is already solved.
Sorry yeah it was a big vague, I was thinking about creating a Libretto MCP since it's a/the standard way to share AI tooling nowadays and that would make it usable in more contexts.
In that case, the protocol has a feature called "sampling" that allow the MCP server (Libretto) to send completion requests to the MCP client (the main agent/harness the user interacts with), that means that Libretto would not need its own LLM API keys to work, it would piggyback on the LLMs configured in the main harness (sampling support "picking" the style of model you prefer too - smart vs fast etc).
Did you consider MCP sampling to avoid requiring your own LLM access? (for the clients that support it of course, but I think it's important and will become standard anyway)
I think sometimes about this, does it really make sense? Financially I mean. The is just my impressions and I'm glad to be corrected if someone has hard numbers and some experience going this route:
At the moment LLMs vendors are in market grab mode and take a loss on big subscription users, they are starting to try to move to profit but they must move carefully to not let a competitor steal their users so we will still have "cheap" tokens for a while.
Even if prices go up by a bit, they have the scale in their favor to optimize costs.
If commercial model providers go into "not competitive" territory with their prices compared to open models, wouldn't it always be cheaper to use an open models inference provider? They can take advantage of scale as well, and with no model moat, competition should keep prices honest.
And last ressort, renting GPU time in the cloud seem like a safer bet than buying a GPU to me?
Yes it's even more effective this way IMO, we will probably see some 11/10 mental gymnastics from people condemning this and failing to apply the same standards to billions dollars corps.
Eh I think part of the joke is that LLMs have gobbled up the original source code, and if you help them enough (identical type signatures and specs), they will output the same code, it's the copyright laundering problem.