From Prototype to Production with the Coasty Computer Use API
Most automation projects stall because they rely on brittle selectors or API-only approaches. The Coasty computer use API solves this by letting you define a task and letting an agent see the screen, reason, and act like a human. You start with a quick prototype using a single predict call, then move to stateful sessions, workflows, and production task runs that drive real desktops and browsers.
How it works
The computer use API is a stateless vision pipeline for online interactions. You send a base64 screenshot, an instruction, and a cua_version. The model returns actions and a status. Loop capture, predict, act until the status is done. For stateful tasks, use POST /v1/sessions to create a trajectory memory, then POST /v1/sessions/{id}/predict to continue. For element targeting, POST /v1/ground maps a screenshot plus a description to x,y coordinates. You can also turn pyautogui code into structured actions with the free POST /v1/parse endpoint.
import os
import base64
import requests
def predict_via_api():
api_key = os.environ.get("COASTY_API_KEY")
url = "https://coasty.ai/v1/predict"
# Replace with your real base64 screenshot
screenshot = "/path/to/screenshot.png"
with open(screenshot, "rb") as f:
img_b64 = base64.b64encode(f.read()).decode("utf-8")
headers = {
"X-API-Key": api_key,
"Content-Type": "application/json"
}
payload = {
"image": img_b64,
"instruction": "Click the button with the text 'Submit' and type '[email protected]' into the email field",
"cua_version": "v3"
}
r = requests.post(url, json=payload, headers=headers)
r.raise_for_status()
result = r.json()
print(result)
if __name__ == "__main__":
predict_via_api()Stateful sessions and element targeting
- ●POST /v1/sessions creates a trajectory memory for multi-step tasks. This is $0.10 per session.
- ●POST /v1/sessions/{id}/predict continues the session with $0.04 per predict call.
- ●POST /v1/ground costs $0.03 and maps a screenshot plus a description to x,y coordinates.
- ●POST /v1/parse is free and turns pyautogui code into structured actions you can reuse.
Start with a stateless predict call for quick prototypes, then adopt sessions and ground/predict for robust, multi-step workflows.
From scripts to production: task runs
For production use, you let the server drive an agent to completion with a task run. POST /v1/runs accepts a machine_id, task, cua_version (default v3, v4 adds a pass/fail verifier), optional instructions, system_prompt, max_steps, deadline_seconds, and a webhook_url. The server drives a real desktop, browser, or terminal. You are billed $0.05 per agent step. GET /v1/runs and GET /v1/runs/{id} show status. GET /v1/runs/{id}/events streams Server-Sent Events; reconnect with Last-Event-ID. States include queued, running, awaiting_human, succeeded, failed, cancelled, and timed_out.
Where this beats brittle automation
Traditional automation relies on XPath, CSS selectors, or API endpoints that change without notice. The computer use API lets you describe what to achieve in natural language, and the agent sees the screen to find the right button, input, or element. This makes your agents resilient to UI changes, layout shifts, and missing APIs. You can target any element visible on the screen, including dynamic content, modals, and native system controls. Coupled with workflows and task runs, you orchestrate complex flows across multiple applications without brittle selectors.
You can prototype a computer use agent in minutes, then scale to production with sessions, workflows, and task runs. Start with a predict call or a free parse, then move to stateful sessions and production task runs that drive real desktops and browsers. Get your API key at https://coasty.ai/developers and start building agents that see and act like humans.