Desktop automation usually means guessing IDs or relying on fragile selectors, but those break when UIs change or apps move elements around. The Coasty computer use API lets you drive real desktops, browsers, and terminals using vision plus action. You send a screenshot, an instruction, and the API returns clicks, keystrokes, and navigation, just like a human. This guide shows how to automate workflows, run agents, and manage machines, with working code you can copy and run.
How it works
The Coasty computer use API works in three layers. First, you provision a machine with POST /v1/machines to get a cloud VM. Then you start a task run with POST /v1/runs, providing a machine_id, a task description, and optional instructions or a system_prompt. The API bills $0.05 per agent step. You can monitor progress with GET /v1/runs/{id} and stream events with GET /v1/runs/{id}/events. If you need structured actions, POST /v1/parse converts pyautogui code into structured actions for free. For vision-only interactions, POST /v1/predict costs $0.05 per call and returns x,y coordinates and actions.
curl -X POST https://coasty.ai/v1/runs \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"machine_id": "machine_abc123",
"task": "Open Chrome, navigate to https://example.com, and click the first heading",
"cua_version": "v4",
"system_prompt": "You are a desktop automation agent. Use clear steps and verify actions."
}'Provision a machine for automation
- Use POST /v1/machines to create a cloud VM with a machine_id
- Start and stop the machine as needed before and after runs
- The agent drives the real desktop UI, not just API calls
- Each machine can run multiple task runs in sequence
1 credit = $0.01. Every agent step costs $0.05.
Run a task with the computer use agent
Start a run with POST /v1/runs. Include machine_id, task, cua_version (v3 or v4), optional instructions, system_prompt, max_steps, deadline_seconds, on_awaiting_human, and webhook_url. The API returns a run_id. Poll GET /v1/runs/{id} for status (queued, running, awaiting_human, succeeded, failed, cancelled, timed_out). Stream events with GET /v1/runs/{id}/events using Last-Event-ID for reconnection. If the run needs human approval, set on_awaiting_human to pause, fail, or cancel. You can also cancel with POST /v1/runs/{id}/cancel or resume with POST /v1/runs/{id}/resume.
import os
import requests
COASTY_API_KEY = os.getenv("COASTY_API_KEY")
BASE_URL = "https://coasty.ai/v1"
def start_run(machine_id, task):
url = f"{BASE_URL}/runs"
headers = {
"X-API-Key": COASTY_API_KEY,
"Content-Type": "application/json",
}
payload = {
"machine_id": machine_id,
"task": task,
"cua_version": "v4",
"max_steps": 50,
"on_awaiting_human": "pause",
}
resp = requests.post(url, headers=headers, json=payload)
resp.raise_for_status()
return resp.json()
run = start_run("machine_abc123", "Open Chrome, visit https://coasty.ai, and click the login button")
print(run)Where this beats brittle automation
Traditional automation relies on stable selectors, which break when UIs change, apps update, or layouts shift. The Coasty computer use API sees the screen like a human, reads text, and clicks visually. It works with any desktop app, browser, or terminal that renders on a desktop. Vision plus action means fewer brittle selectors, less maintenance, and automation that adapts to UI changes. You can also build workflows with POST /v1/workflows, version a DSL of steps, and use variables, loops, and conditions for complex, reusable automation.
You can now automate any desktop app with the Coasty computer use API. Start by provisioning a machine, then run task steps or workflows. Monitor progress with runs and events. Get a key at https://coasty.ai/developers to begin building reliable desktop automation right now.
Want to see this in action?
View Case Studies