HackerTrans
TopNewTrendsCommentsPastAskShowJobs

cjbell88

33 karmajoined hace 14 años

Submissions

Show HN: Email API benchmarks – Real-world performance data for email providers

knock.app
8 points·by cjbell88·hace 4 meses·3 comments

comments

cjbell88
·hace 4 días·discuss
Yep we built our own harness from scratch, directly on the Anthropic API
cjbell88
·hace 4 días·discuss
Author here. This post covers how we rebuilt our agent after a tool-per-resource approach didn't scale — we replaced most of our tools with a virtual filesystem the agent explores via bash.

A few things that might be interesting to discuss:

- We didn't want to boot a container per session, so we run a bash interpreter + virtual FS in-memory as a process in our Elixir cluster. This is a port of Vercel's just-bash (TypeScript) to Elixir. The original's test suite made it a well-defined target for an agent-assisted port — we reused the fixtures verbatim: https://github.com/elixir-ai-tools/just_bash

- The "why not a real sandbox" tradeoff is the one I'm least certain about long-term. In-memory gets us instant starts and no sync problems, but if we add a real scripting language (Python) for the agent, we'll probably have to swap in a real sandbox. We've kept the interfaces decoupled so that swap stays cheap.

- For data that doesn't map well to static files (logs, message history), we registered a `knock` CLI inside the bash environment instead of adding more tools. Because it has --help for every resource, the agent learns it with almost no steering.

Happy to answer questions about the architecture, the Oban-backed agent loop, or how we're evaluating trajectories (nightly evals, pass^k on common tasks).
cjbell88
·hace 4 días·discuss
[flagged]