I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.
Thanks. That’s a fair comparison, but I’m thinking a bit broader than model serving alone.
I’m less interested in building a generic inference backend and more focused on how the system behaves over time — things like agent lifecycles, state, memory, and coordination. Inference is part of it, but not the whole thing.
Correctness, performance, and privacy matter because the system is long-lived and stateful, not just because it’s pushing tokens around.
Yes — definitely interested in the client side, especially where it intersects with core system design rather than just presentation.
Both native markdown rendering and non-linear / branching conversation models are areas I’ve been thinking about as well, particularly as systems move beyond linear chat and into more exploratory or agent-driven workflows.
Would be curious to hear more about what you experimented with and what tradeoffs you ran into, especially on the data model and UI side.
I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.