HackerTrans
TopNewTrendsCommentsPastAskShowJobs

cjbell88

33 karmajoined 14 năm trước

Submissions

Show HN: Email API benchmarks – Real-world performance data for email providers

knock.app
8 points·by cjbell88·4 tháng trước·3 comments

comments

cjbell88
·4 ngày trước·discuss
Yep we built our own harness from scratch, directly on the Anthropic API
cjbell88
·5 ngày trước·discuss
Author here. This post covers how we rebuilt our agent after a tool-per-resource approach didn't scale — we replaced most of our tools with a virtual filesystem the agent explores via bash.

A few things that might be interesting to discuss:

- We didn't want to boot a container per session, so we run a bash interpreter + virtual FS in-memory as a process in our Elixir cluster. This is a port of Vercel's just-bash (TypeScript) to Elixir. The original's test suite made it a well-defined target for an agent-assisted port — we reused the fixtures verbatim: https://github.com/elixir-ai-tools/just_bash

- The "why not a real sandbox" tradeoff is the one I'm least certain about long-term. In-memory gets us instant starts and no sync problems, but if we add a real scripting language (Python) for the agent, we'll probably have to swap in a real sandbox. We've kept the interfaces decoupled so that swap stays cheap.

- For data that doesn't map well to static files (logs, message history), we registered a `knock` CLI inside the bash environment instead of adding more tools. Because it has --help for every resource, the agent learns it with almost no steering.

Happy to answer questions about the architecture, the Oban-backed agent loop, or how we're evaluating trajectories (nightly evals, pass^k on common tasks).
cjbell88
·5 ngày trước·discuss
[flagged]