Most UI automation scripts break after one click. You click a button, the DOM changes, and your brittle selector fails. The computer use API solves this by letting the model see the screen and act like a human. Two patterns exist. The stateless predict loop sends a screenshot and instruction each time. The stateful session pattern creates a session, then posts a predict call that receives the full trajectory of past actions. The session keeps memory between steps so the model understands context. Use stateless predict for single-shot tasks. Use stateful sessions for multi-step workflows like checkout flows, form fills, or multi-tab browser tasks.
How it works
The stateless flow is simple. You POST /v1/predict with a base64 screenshot, an instruction, and cua_version. The endpoint returns actions and a status. You capture a new screenshot, post again, and repeat until status is done. Each predict costs $0.05. This works when the task completes in one round of inference. The stateful flow adds a session. First, POST /v1/sessions with a task, max_steps, deadline_seconds, and optional instructions. The response includes a session_id. Then POST /v1/sessions/{id}/predict with the current screenshot and instruction. The response again includes actions and status, but the server also stores the trajectory in the session. Future predict calls can read that history, so the model knows what you clicked before, what UI elements appeared, and the current state. The per-step price drops to $0.04. Sessions also support cancellation and resumption, which is useful for long-running workflows.
#!/bin/bash
# Stateless predict loop (single screenshot, no session)
export COASTY_API_KEY
SCREENSHOT="$(base64 -w 0 screenshot.png)"
INSTRUCTION="Find the search bar on this page and type 'coasty' then hit Enter."
while true; do
RESPONSE=$(curl -s https://coasty.ai/v1/predict \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"screenshot\": \"$SCREENSHOT\", \"instruction\": \"$INSTRUCTION\", \"cua_version\": \"v3\"}")
echo "$RESPONSE" | jq .
STATUS=$(echo "$RESPONSE" | jq -r '.status')
if [ "$STATUS" = "done" ]; then
break
fi
# Capture new screenshot for next iteration
SCREENSHOT="$(base64 -w 0 screenshot.png)"
doneimport os
import base64
import requests
import json
COASTY_API_KEY = os.environ.get("COASTY_API_KEY")
BASE_URL = "https://coasty.ai/v1"
# Stateful session creation
session_resp = requests.post(
f"{BASE_URL}/sessions",
headers={"X-API-Key": COASTY_API_KEY},
json={
"task": "Navigate to cozy.ai and click the login button, then fill in a test email address.",
"max_steps": 20,
"deadline_seconds": 60,
"cua_version": "v4"
}
)
session_resp.raise_for_status()
session_id = session_resp.json()["session_id"]
print("Session created:", session_id)
# Stateful predict loop
screenshot = base64.b64encode(open("screenshot.png", "rb").read()).decode()
instruction = "Click the login button on this page."
while True:
predict_resp = requests.post(
f"{BASE_URL}/sessions/{session_id}/predict",
headers={"X-API-Key": COASTY_API_KEY},
json={
"screenshot": screenshot,
"instruction": instruction,
"cua_version": "v4"
}
)
predict_resp.raise_for_status()
result = predict_resp.json()
print("Result:", json.dumps(result, indent=2))
status = result.get("status")
if status == "done":
break
screenshot = base64.b64encode(open("screenshot.png", "rb").read()).decode()Session lifecycle
- Create a session with POST /v1/sessions (returns session_id).
- Keep the session_id across predict calls to maintain trajectory memory.
- Each predict call on a session costs $0.04, compared to $0.05 for stateless.
- Use GET /v1/sessions/{id} to inspect session state or history.
- Cancel a long-running session with POST /v1/sessions/{id}/cancel.
- Resume a paused session with POST /v1/sessions/{id}/resume.
Stateful sessions keep trajectory memory between steps, lowering per-step cost and enabling multi-step workflows.
When to use stateless predict
Stateless predict is the right choice when a single round of inference can complete the task. Examples include clicking a single button, submitting a simple form, or running a quick assertion. You avoid the overhead of session creation and cancellation. Each predict costs $0.05, but you only pay for as many calls as the workflow needs. This pattern is ideal for brittle automation where you want to treat each interaction as independent. You still get the benefits of computer use, vision-based actions, but you do not need the server to store history across calls.
Where this beats brittle automation
Traditional automation relies on selectors, XPath, or API mocks. If the UI changes, the script breaks. Computer use agents see the screen. They click where you point and type what you ask. Stateful sessions let the agent remember the sequence of actions and the current state, so it can adapt when the UI shifts. You do not need to maintain brittle selectors or rewrite scripts for every layout change. You just describe the goal in natural language, and the agent navigates the real UI. This approach works for web browsers, desktop apps, and terminals.
Choose stateless predict for simple tasks. Use stateful sessions for multi-step workflows that need trajectory memory. Get a key at https://coasty.ai/developers to start building reliable computer use agents today.
Want to see this in action?
View Case Studies