ya i do definitely agree that learning Vim is gonna help my overall understanding for how things work at a deeper level and also fight back a lot of the “learned helplessness” that i did develop when coding w/ AI to your point
also another thing that i was thinking is that yes short term (maybe the first few months?) i wouldn’t see any benefit… but it would definitely help in the long term and that my coding ability is not just directly tied to whatever the latest model is capable of
Interesting! First time im seeing this course, thanks for the link. From a high level it’s definitely in the same code first agents family then. After reading about smolagents for a bit i think the main things Codecall adds are TypeScript + generated SDKs, progressive tool discovery (readFile + executeCode instead of exposing every tool directly), and the single script sandboxed execution first flow w/ learned constraints, rather than the more of the "multi‑step ReAct loop" that smolagents prioritizes (like in the link below), which is a bit more like traditional tool calling w/ code ->
Traditional AI agents have EVERY tool loaded into context from the stat, call tools one at a time, each requiring a full inference round trip, for example: "delete all completed tasks," that means: call findTasks, wait, call deleteTask for task 1, wait, call for task 2... each call resends the entire conversation history, so tokens compound fast and there is a lot of wasted tokens and inference.
Codecall is an open source approach that lets agents write and execute TypeScript code in a secured Deno sandbox to orchestrate multiple tools programmatically, like calling an API (which is really all a tool is!)
So instead of 20+ inference passes and 90k+ tokens, the agent can just write and execute:
const tasks = await tools.todoist.findTasks({ completed: true });
for (const task of tasks) {
await tools.todoist.deleteTask({ id: task.id });
}
2 inference passes. The code runs in a Deno sandbox, executes all operations programmatically, and returns a result. In our demo, for one example, this reduced tokens by 74.7% and tool calls by 92.3% while being much faster as well.
How it works (high level) ->
1. There are only 2 tools (readFile, executeCode) + a file tree. The agent reads SDK files on demand, so a 30 tool setup is effectively the to a 5 tool setup (only the file tree gets bigger)
2. Multiple tool calls happen in one execution, not N inference calls for N operations... because the agent can write code to execute and orchestrate multiple tools (like API) this significantly reduces the number of passes + tokens per request
3. Models have a 10-50% failure rate searching through large datasets in context. Code like users.filter(u => u.role === "admin") is deterministic and avoids those failure, so not only is it more efficient & cheaper. its also often much more accurate when doing operations with lots of data!
We also generate TypeScript SDK files from MCP tool definitions, so the agent sees clean types and function signatures. It also learns from errors, so when a tool call fails, it updates the SDK file with learned constraints so future agents avoid the same mistake.
Codecall works with any MCP server (stdio/http). Would love feedback from anyone interested in or building more complex agents :)
Sounds cool man! Open sourcing the core while offering a hosted service (like a PostHog model) would for sure build trust fast, and then monetize a hosted service for users who prefer a managed solution (most people).. i'd be happy to contribute if you go that route :)
Not exactly sure how big the market is but i would for sure always go the B2B route, imo most consumers will stick to whatever desktop tools they currently use.
I've been working on an open source implementation of Programmatic Tool Calling for Agents, based on cloudflare's codemode & a few anthropic articles, and although i think it can be very powerful in certain usecases, there are some challenges that i would love to have your thoughts on
Instead of traditional agents that burn tens of thousands of tokens loading all tool definitions upfront and compound context with sequential calls, this approach lets agents discover only the tools they need from a file tree of TypeScript SDKs, then write code to one-shot tasks in a single pass.
Although having an agent execute code seems like its ideal as LLMs are great at writing code, there are a few big challenges that i have faced below
The main challenges w/ Programmatic Tool Calling:
- Output Schemas from the Tools
MCP servers or most tool definitions almost never define output schemas, and without knowing what a tool returns, the model hallucinates property names, like think of 'task.title' vs 'task.name' as an example, and the script fails at runtime because it has too guess the shape of the output of a tool. I'm working around this by the classifying tools and by actually calling the tools to infer schemas, but it's really hacky because a single sample misses optional fields, and testing write + destructive tools means creating real or destroying data which is an approach i really dislike and don't think is viable
- Tool Outputs Are Often Plain Strings (returns unstructured data)
Even with perfect schemas and defined shapes, most MCP tools return markdown blobs or plain strings meant for LLM inference. No JSON, no fields to index into and just text. If majority of your tools return in just strings (even when listing data) the main value of codecall is lost because you can't write deterministic code against unstructured data in a string. You're forced back into traditional agent behavior where the LLM interprets text. If you don't control the server or the tool definitions, there's no fix i can really think of.
- Input/Output examples for each Tool (Amplified w/ Programmatic Tool Calling)
The final challenge is that JSON Schema defines structure but not usage patterns. Take that support ticket API example: the schema tells you due_date is a string, but not whether it wants "2024-11-06" or "Nov 6, 2024". It says reporter.id is a string, but is that a UUID or "USR-12345"? When should reporter.contact be populated? How do escalation.level and priority interact? (got this example from an anthropic article covering this)
In traditional tool calling, the model can learn these patterns through trial and error across multiple turns. It tries something, gets an error or unexpected result, and adjusts for the rest But with programmatic tool calling, the model writes a script that might call create_ticket 50 times in a loop for different users. If it misinterprets the date format or ID convention in the first call, all 50 calls fail and so on.
-------------
Although all of these could be fixed by just setting them manually by the user, is there a reliable way we can get the Output Schemas and generate Input/Output examples for each Tool, without actually calling the tool, and without having a user manually input the data?
If anybody is interested, or has any thoughts on Tool Calling for Agents and has any ideas please feel free to share!