Building AI coding tools, there are two common options.
The AI suggests code and you run it. Or the AI runs code on your actual machine. We decided to go with the third option: The AI gets its own computer.
Sandboxed Linux workspaces that CoChat LLMs spin up on demand.
It Write code, installs dependencies, runs tests and start servers.
We are now running an automation that gets triggered every time an error is logged in Grafana Labs, the agent then spins up the workspace to verify and fix the issue before creating a PR and Issue on GitHub. If it had already addressed the issue previously it will just track occurrences.
There is something to be said about that, I agree. For that reason you can turn off memory inside a chat thread and also create temporary ones that do not use memory.
Thank you and great question. Right now, feedback is qualitative only. (Surveys, feedback buttons, controlled user tests). We are trying to build AI evaluators but they suffer from the same problem
when trying to evaluate whether the “right” memory was pulled.
The AI suggests code and you run it. Or the AI runs code on your actual machine. We decided to go with the third option: The AI gets its own computer.
Sandboxed Linux workspaces that CoChat LLMs spin up on demand. It Write code, installs dependencies, runs tests and start servers.
We are now running an automation that gets triggered every time an error is logged in Grafana Labs, the agent then spins up the workspace to verify and fix the issue before creating a PR and Issue on GitHub. If it had already addressed the issue previously it will just track occurrences.
We get notified on Slack.