You can prototype a UI automation agent in minutes, then scale it to production with cloud machines and workflows. This guide shows how to move from a single-page task to a robust, repeatable computer use agent using Coasty.
How it works
The Coasty computer use API lets you build agents that see the screen, interpret instructions, and act like a human. The core flow is capture -> predict -> act. You send a screenshot and instruction to /v1/predict, get back a list of actions (click, type, scroll, etc.), then capture again until status is done. For stateful workflows, use /v1/sessions/{id}/predict which keeps the trajectory in memory. To locate elements by description, call /v1/ground with a screenshot and element description and get back x,y coordinates. You can also convert any PyAutoGUI script into structured actions with the free /v1/parse endpoint.
#!/bin/bash
# Example: Run a simple click task with the Coasty computer use API
# Requires COASTY_API_KEY environment variable
URL="https://coasty.ai/v1"
KEY=$(echo "$COASTY_API_KEY")
# 1. Create a task run (server-driven agent)
curl -X POST "$URL/runs" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{
"machine_id": "m1",
"task": "Open https://example.com and click the first link that contains the word test",
"cua_version": "v4",
"max_steps": 20,
"deadline_seconds": 60,
"on_awaiting_human": "pause"
}' | jq .Prototype with a single page
- Use /v1/predict to send a screenshot and instruction, paying $0.05 per call.
- Loop capture, predict, act until status is done.
- Start with simple tasks like clicking a button or typing into an input.
- Read the events stream with GET /v1/runs/{id}/events to see the agent's actions in real time.
Scale to production with cloud machines
- Provision a cloud machine with POST /v1/machines, providing machine_id and any specs.
- Use the same task and cua_version on the cloud machine.
- Manage machine lifecycle with start, stop, and snapshot operations.
- Each agent step costs $0.05 per task run. Workflows let you orchestrate multiple tasks, asserts, loops, and retries in one DSL.
Orchestrate with workflows
- Define a versioned JSON DSL with POST /v1/workflows.
- Include steps like task, assert, if, loop, parallel, human_approval, retry, succeed, fail.
- Use variables like {{inputs.x}} and stepId.field to reference values across steps.
- Define hard guards like budget_cents, max_iterations, and deadline_seconds.
- Each task step within the workflow is billed $0.05 per agent step.
Start with /v1/predict for fast prototyping, then move to /v1/runs for server-driven agents and /v1/workflows for complex orchestration.
Where this beats brittle automation
Traditional selectors and API-only tools break when UI changes, layouts shift, or elements move. The Coasty computer use API drives the agent like a human: it sees the actual screen, reads text, and acts on context. This means your automation stays resilient to UI changes, supports complex interactions, and can handle dynamic content without brittle selectors.
You can prototype a UI automation agent in minutes, then scale it to production with cloud machines and workflows. Get your API key at https://coasty.ai/developers and start building computer use agents that see, understand, and act.
Want to see this in action?
View Case Studies