From Prototype to Production with the Coasty Computer Use API
You start with a quick prototype that clicks a button in a browser. Then you need to run it on real machines, handle changing UI, and pay only for steps that actually execute. The Coasty computer use API lets you build an agent that sees the screen and acts like a human, not a brittle selector list. Use real APIs for stateful sessions, task runs, and workflows to go from a one‑off script to a production system.
Core concepts and endpoints
The API revolves around three main models. First, a vision model that takes a screenshot and instruction and returns actions. POST /v1/predict takes a base64 screenshot, an instruction, and cua_version. It costs $0.05 per call and returns actions plus a status. Loop capture, predict, act until status is done. Second, stateful sessions let you keep memory across calls. POST /v1/sessions creates a session, then POST /v1/sessions/{id}/predict adds a screenshot and instruction. This second predict step costs $0.04 and carries the session’s trajectory. Third, task runs let the server drive an agent to completion. POST /v1/runs accepts machine_id, task, cua_version (v3 or v4), instructions, system_prompt, max_steps, deadline_seconds, on_awaiting_human, and webhook_url. Each agent step in a run costs $0.05. You can query, cancel, or resume runs with GET /v1/runs, GET /v1/runs/{id}, POST /v1/runs/{id}/cancel, and POST /v1/runs/{id}/resume. Events stream via GET /v1/runs/{id}/events with Server‑Sent Events.
A working prototype in Python
- ●Install requests and a base64 encoder (Python 3 includes base64).
- ●Read COASTY_API_KEY from the environment for security.
- ●Use the predict endpoint to turn a screenshot and instruction into actions.
- ●Loop until the status field is done.
import base64
import os
import requests
COASTY_API_KEY = os.environ.get('COASTY_API_KEY')
BASE_URL = 'https://coasty.ai/v1'
# Encode a raw PNG screenshot to base64 (replace with your own)
with open('screenshot.png', 'rb') as f:
screenshot_b64 = base64.b64encode(f.read()).decode('utf-8')
# Example: click on a button that says 'Open Settings'
payload = {
'screenshot': screenshot_b64,
'instruction': 'Click the button that says Open Settings.',
'cua_version': 'v3'
}
headers = {'X-API-Key': COASTY_API_KEY}
url = f'{BASE_URL}/predict'
response = requests.post(url, json=payload, headers=headers)
response.raise_for_status()
result = response.json()
print('Actions:', result.get('actions'))
print('Status:', result.get('status'))
# Loop until status is done (for completeness)
while result.get('status') != 'done':
# Capture new screenshot and repeat
with open('screenshot.png', 'rb') as f:
screenshot_b64 = base64.b64encode(f.read()).decode('utf-8')
payload['screenshot'] = screenshot_b64
response = requests.post(url, json=payload, headers=headers)
response.raise_for_status()
result = response.json()
print('Actions:', result.get('actions'))
print('Status:', result.get('status'))Stateful sessions for memory
A single predict call is fine for one shot. For longer tasks, sessions keep trajectory memory. First POST /v1/sessions to get an id. Then repeatedly POST /v1/sessions/{id}/predict with the screenshot and instruction. This second predict step costs $0.04. The server retains the session’s history so the model can reason across steps. You can inspect the session’s trajectory to debug or feed it into downstream tools.
Task runs for complete automation
- ●POST /v1/runs with machine_id and task to launch an agent.
- ●cua_version can be v3 (guided) or v4 (autonomous with pass/fail).
- ●Each agent step in the run costs $0.05.
- ●Events stream via GET /v1/runs/{id}/events for real‑time status.
Workflows for multi‑step orchestration
Workflows are a versioned JSON DSL that composes runs. POST /v1/workflows creates a workflow. POST /v1/workflows/{id}/runs launches it. You can also submit inline with POST /v1/workflows/runs. Step types include task, assert, if, loop, parallel, human_approval, retry, succeed, and fail. Conditions and variables let you guard with budget_cents, max_iterations, and deadline_seconds. Task steps are billed $0.05 each.
POST /v1/runs with machine_id and task is the entry point for production agents.
Where this beats brittle automation
Traditional automation relies on selectors that break when a UI element changes position or class name. A computer use agent sees the screen and reads text, so it adapts to layout shifts. It can handle dynamic elements, tooltips, and multi‑step flows without extra glue code. You pay only for actual steps that the agent executes, not for idle time. Combine stateful sessions for memory, task runs for end‑to‑end automation, and workflows for complex pipelines.
Start with a prototype using POST /v1/predict. Move to stateful sessions when you need memory, then to task runs and workflows for production scale. Deploy on real machines with POST /v1/machines and monitor events via Server‑Sent Events. Build reliable agents that adapt to UI changes instead of brittle selectors. Get a key at https://coasty.ai/developers and start building.