Launch HN: Coasty (YC S26) – An API for computer-use agents(coasty.ai)
coasty.ai
Launch HN: Coasty (YC S26) – An API for computer-use agents
https://coasty.ai/docs
15 コメント
The most interesting thing about this is buried in a reply rather than the post: the ability to blend deterministic workflows with AI calls, so the agent kicks in on recovery when a dialog or popup breaks the scripted path.
That's your actual differentiator and I'd lead with it. Right now the post frames it as observe-decide-execute-observe, which reads as "another CUA agent," and the top comment is immediately asking how you differ from the other CUA APIs in the batch. But deterministic-by-default with AI on exception is a genuinely different shape from both of the alternatives you name. RPA breaks when the screen changes; a pure agent is nondeterministic on the 95% of steps that never needed judgment in the first place. Doing the boring path deterministically and paying for a model only when reality deviates is the thing that would make me try this.
I came at the same conclusion from a completely unrelated domain (I build asset generators, no computer-use anywhere near it) and the tool only got good the day I stopped letting the model do the parts that had to be identical every run. Generation is sampling. Same prompt, same model, different run, different result. The skill is knowing which steps genuinely need judgment and refusing to spend a sampling operation on the ones that don't. Sounds like you've built exactly that control surface and then under-sold it.
Genuine question on the handoff, since that's where I'd expect this to get hard: when the deterministic path hits something unexpected and the agent takes over for recovery, how does it hand control back? Does it have to return the UI to a known state that the script recognizes, or does the script resume wherever the agent left off? That reconciliation seems like the interesting engineering, and it's also where I'd expect the failure modes to live.
That's your actual differentiator and I'd lead with it. Right now the post frames it as observe-decide-execute-observe, which reads as "another CUA agent," and the top comment is immediately asking how you differ from the other CUA APIs in the batch. But deterministic-by-default with AI on exception is a genuinely different shape from both of the alternatives you name. RPA breaks when the screen changes; a pure agent is nondeterministic on the 95% of steps that never needed judgment in the first place. Doing the boring path deterministically and paying for a model only when reality deviates is the thing that would make me try this.
I came at the same conclusion from a completely unrelated domain (I build asset generators, no computer-use anywhere near it) and the tool only got good the day I stopped letting the model do the parts that had to be identical every run. Generation is sampling. Same prompt, same model, different run, different result. The skill is knowing which steps genuinely need judgment and refusing to spend a sampling operation on the ones that don't. Sounds like you've built exactly that control surface and then under-sold it.
Genuine question on the handoff, since that's where I'd expect this to get hard: when the deterministic path hits something unexpected and the agent takes over for recovery, how does it hand control back? Does it have to return the UI to a known state that the script recognizes, or does the script resume wherever the agent left off? That reconciliation seems like the interesting engineering, and it's also where I'd expect the failure modes to live.
The checkpoint and invariant model is a strong fit for these workflows. Having approval gates plus a replayable event log makes the agent's decisions much easier to audit than a simple end-to-end task API.
How are you all different than the other few CUA APIs in this batch and previous batches?
We don't just provide end-to-end API, we're a SOTA harness where you can bring in your own OpenAI or Claude keys and run it on and also we're the only modular API in the market - we give you the ability to control any part of CUA and charge by the type of call as you can see in https://coasty.ai/docs.
What do you think that makes your solution "SOTA"? That's quite the interesting claim, which is obviously false.
That’s really nice actually. Do the screenshots and playback and logs get stored on our systems as well?
Yes, you have access to all your logs and screenshots and playback!
Sure but is that stored on our infra or yours? (Work in HIPAA regulated industry)
We're SOC2 and HIPAA compliant and have zero data retention policies through our enterprise platform where it's all covered and we don't keep any data that you don't want us to!
With our API, you can also create a very customizable blend of deterministic workflows and AI calls for CUA, so for example, Coasty for recovery can kick in when something unexpected like a dialog box or popup happens during your deterministic workflow runs.
It's so funny to see YC back tens of generic similar low quality projects
[deleted]
Developers send Coasty a natural-language task either through our consumer app or through our API, select a machine or browser environment, and any relevant credentials or files. The agent then operates the interface through screenshots, mouse, and keyboard input, verifies the result, and returns a structured run record with screenshots, actions, outputs, and errors.
Here is a raw demo of an agent completing a workflow in a legacy application(It’s a mockup): https://drive.google.com/file/d/1ZghU_3vsAYhHVz1bsvE0pkvZYk7...
A lot of important software is still difficult to automate. Healthcare teams submit prior authorizations through payer portals, accounting teams enter data into desktop applications, and operations teams move information between internal systems, spreadsheets, and remote desktops. Many of these applications have no API, incomplete APIs, or integrations that take months to build.
The usual alternative is RPA, record a sequence of clicks and replay it. That works when the interface and workflow are predictable, but it often breaks when a button moves, a pop-up appears, a page loads slowly, or the application enters an unexpected state.
Coasty takes a different approach. The agent observes the current screen, decides what action to take, executes it, and then observes the resulting state before continuing. It does not require DOM access, an accessibility tree, selectors, or an application-specific integration, so the same API can operate browsers, remote desktops, and older Windows applications.
A simplified request looks roughly like this:
The response includes the final status, extracted outputs, a replay URL, and a timestamped event log:
The API can also pause a run for human approval, retry from a checkpoint, or return control to the developer when it encounters a condition the workflow did not anticipate.
We started working on this last summer, because we saw that models were getting better at vision but kept seeing a gap between computer-use demos and the reliability needed for production workflows. Getting an agent to complete a task once is fairly straightforward. Getting it to repeat that task, recover from unexpected states, avoid silently entering incorrect data, and produce evidence of what it did is much harder.
We built several layers around the underlying computer-use model. The system tracks the expected state of the workflow, detects when the application has diverged from that state, and can re-plan instead of continuing blindly. Developers can define invariants such as “the patient name must match the source document” or “never submit without approval,” and the agent checks those conditions during the run.
Each run happens in an isolated virtual machine. We expose APIs for provisioning environments, uploading files, starting tasks, streaming events, inserting human approvals, and retrieving the full replay and audit trail. Environments can be kept alive across runs when the application has a long login flow or persistent local state.
One problem we are still working through is the tradeoff between speed and reliability. The agent can move faster by taking fewer observations and verification steps, but that becomes risky in workflows involving patient records, payments, or regulatory filings. We currently bias toward slower execution with more checks and let developers configure approval points and verification policies.
We are initially working with healthcare operations teams because their workflows combine many of the hardest conditions: payer portals, EHRs, PDFs, spreadsheets, remote desktops, and actions where quiet mistakes are expensive. We also expose the same infrastructure through the developer API for teams building their own agents and vertical automation products.
We currently charge based on agent runtime and workflow volume, with separate pricing for dedicated environments and enterprise deployments.
We’d especially appreciate feedback from people who have built and/or used browser agents, RPA systems, desktop automation, or agent infrastructure. We’re curious which parts of the API you would want direct control over, where you would prefer higher-level abstractions, and which failure modes have been hardest in your own automation systems.
If you've hit weird failure modes automating software like this, we want to hear about them. We'll be here all day answering questions and taking notes!