Most teams start with a simple automation script that clicks hardcoded X,Y coordinates or uses brittle selectors. That works for a prototype but breaks fast when layouts change or pages load dynamically. The Coasty computer use API lets you move directly to production by providing a vision-based agent that watches the screen, understands instructions, and takes actions just like a human. You can start with a prototype and scale to complex workflows. This guide shows the exact workflow for going from a first call to a production-ready system.
What you need to know before you start
- Read your API key from the COASTY_API_KEY environment variable. Do not hardcode it.
- Base URL is https://coasty.ai/v1. Use the X-API-Key header with your key or Bearer authorization.
- Task Runs are billed $0.05 per agent step. A session predict costs $0.04 and a vision predict costs $0.05.
- You can stream events with GET /v1/runs/{id}/events to see state changes in real time.
How it works
The production workflow consists of three main stages. First you create stateful sessions to keep track of the environment. Then you drive the agent through tasks using predict calls that accept a base64 screenshot, an instruction, and the CUA version. Finally you monitor the run state using events and webhooks. The server drives the agent to completion, sending status updates through the event stream. You can cancel or resume runs if needed. Here is an example that starts a session, sends a simple instruction, and waits for completion.
import base64
import os
import requests
from typing import Optional
def encode_image(path: str) -> str:
with open(path, "rb") as f:
return base64.b64encode(f.read()).decode("utf-8")
def run_simple_task(
instruction: str,
machine_id: str,
cua_version: str = "v3",
max_steps: Optional[int] = None,
) -> dict:
api_key = os.getenv("COASTY_API_KEY")
if not api_key:
raise ValueError("COASTY_API_KEY environment variable must be set")
base_url = "https://coasty.ai/v1"
headers = {"X-API-Key": api_key}
# 1. Create a stateful session
session_resp = requests.post(
f"{base_url}/sessions",
headers=headers,
json={
"machine_id": machine_id,
"cua_version": cua_version,
},
)
session_resp.raise_for_status()
session = session_resp.json()
session_id = session["id"]
# 2. Send the first predict call (screenshot + instruction)
predict_resp = requests.post(
f"{base_url}/sessions/{session_id}/predict",
headers=headers,
json={
"screenshot": encode_image("screen.png"),
"instruction": instruction,
"cua_version": cua_version,
},
)
predict_resp.raise_for_status()
result = predict_resp.json()
# 3. Poll until done (example with a simple loop; real code would use events)
while result.get("status") not in ("done", "failed", "cancelled"):
if max_steps and max_steps <= 0:
break
predict_resp = requests.post(
f"{base_url}/sessions/{session_id}/predict",
headers=headers,
json={
"screenshot": encode_image("screen.png"),
"instruction": result.get("next_instruction") or instruction,
"cua_version": cua_version,
},
)
predict_resp.raise_for_status()
result = predict_resp.json()
max_steps = max_steps - 1 if max_steps else None
return resultCreate a stateful session first, then call POST /v1/sessions/{id}/predict in a loop until status is done.
Handling task runs and events
For long-running tasks you can use Task Runs instead of manual sessions. POST /v1/runs creates a run with a machine_id, task, and optional instructions, system_prompt, max_steps, deadline_seconds, and a webhook_url. The server drives the agent and streams events to your endpoint. States include queued, running, awaiting_human, succeeded, failed, cancelled, and timed_out. You can cancel or resume runs using POST /v1/runs/{id}/cancel and POST /v1/runs/{id}/resume. This is ideal for production systems that need fault tolerance and observability.
Where this beats brittle automation
Traditional automation assumes fixed selectors and stable DOM structures. When a layout shifts or a page loads asynchronously, selectors break and scripts fail. The Coasty computer use agent watches the actual screen, understands natural language instructions, and performs actions via keyboard and mouse. It works with real browsers, desktop applications, and terminals. You do not need to maintain selectors or wait for stable APIs. This lets you ship agents that adapt to real-world UI changes and handle complex workflows that are impractical with pure script automation.
You can now move from a prototype to a production computer use agent. Start with stateful sessions for quick experiments, then switch to Task Runs for long-running workflows and event streaming. Use webhooks for notifications and integrate with your monitoring stack. Get your API key and start building at https://coasty.ai/developers. The Coasty computer use API gives you the tools to build agents that truly understand and interact with real interfaces.
Want to see this in action?
View Case Studies