You have a workflow that works in the browser or on the desktop. Now you need an agent to run it reliably in the cloud. A computer use API gives you eyes and hands. You send a screenshot and an instruction. The API returns actions you replay locally or in a cloud VM. You iterate fast, then move to production with stateful sessions, task runs, and workflows.
How the Coasty computer use API works
The core interaction is a predict call with a screenshot and an instruction. Use the vision endpoint POST /v1/predict. It costs $0.05 per prediction. The request requires a base64 screenshot, an instruction string, and the cua_version. The response includes actions, a status, and a request_id. When status is done, the agent has finished the step. You loop capture, predict, act until all steps are done. For longer tasks, start a session with POST /v1/sessions to retain trajectory memory. Then POST /v1/sessions/{id}/predict for $0.04 per step. You can also ground element locations with POST /v1/ground ($0.03).
#!/usr/bin/env bash
set -euo pipefail
COASTY_API_KEY=${COASTY_API_KEY:-}
if [[ -z "$COASTY_API_KEY" ]]; then
echo "Error: COASTY_API_KEY env var required" >&2
exit 1
fi
# Capture a screenshot of the active window and encode to base64
SCREENSHOT_B64=$(python3 -c "
import base64, subprocess, sys
try:
img = subprocess.check_output(['screencapture', '-x', '/tmp/screenshot.png'])
print(base64.b64encode(img).decode('utf-8'))
except Exception as e:
print(f'Error capturing screenshot: {e}', file=sys.stderr)
sys.exit(1)")
# Example instruction: click the first visible button
INSTRUCTION='Click the first visible button.'
# Call the vision endpoint
RESPONSE=$(curl -s -X POST https://coasty.ai/v1/predict \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"screenshot\": \"$SCREENSHOT_B64\",
\"instruction\": \"$INSTRUCTION\",
\"cua_version\": \"v3\"
}")
# Extract the status and actions
STATUS=$(echo "$RESPONSE" | jq -r '.status')
ACTIONS=$(echo "$RESPONSE" | jq -r '.actions')
REQUEST_ID=$(echo "$RESPONSE" | jq -r '.request_id')
# If the agent is still running, loop until done
while [[ "$STATUS" != "done" ]]; do
# In a real flow you would send the actions to your UI/OS
sleep 0.5
# Re-capture and predict again...
done
echo "Request ID: $REQUEST_ID"
echo "Status: $STATUS"
echo "Actions: $ACTIONS"Running full tasks with task runs
- Use POST /v1/runs for server-driven agent execution.
- Provide machine_id, task, cua_version, and optional instructions.
- Set max_steps and deadline_seconds to control cost and runtime.
- Choose on_awaiting_human to pause, fail, or cancel when the agent asks for help.
- Configure a webhook_url to receive status events and results.
- Billed $0.05 per agent step.
- States include queued, running, awaiting_human, succeeded, failed, cancelled, timed_out.
POST /v1/runs is the simplest route from prototype to production when you want the server to drive the agent to completion.
Managing state and events
- GET /v1/runs lists your runs.
- GET /v1/runs/{id} shows the final status and outcome.
- GET /v1/runs/{id}/events streams Server-Sent Events for real-time progress.
- Reconnect with the Last-Event-ID header to resume reading after a disconnect.
- Use request_id to correlate events to your application logs.
Where this beats brittle automation
Traditional tools rely on selectors, XPath, or hardcoded element IDs. Those break when UI changes or elements are dynamically generated. A computer use agent sees the screen. It follows instructions in natural language. It can click, drag, type, and scroll. It adapts to layout shifts. You iterate at the level of behavior, not DOM selectors. This is especially valuable for workflows that span multiple applications, non-browser UI, or when you need to support rapid UI changes without maintaining selectors.
Advanced orchestration with workflows
- Define workflows as a versioned JSON DSL using POST /v1/workflows.
- Combine tasks, asserts, if conditions, loops, parallel steps, human approvals, retries, and succeed/fail outcomes.
- Use variables like {{inputs.x}} and stepId.field for dynamic values.
- Hard guards such as budget_cents, max_iterations, and deadline_seconds keep runs safe.
- Task steps in workflows are billed $0.05 each.
Start with a simple predict loop to iterate quickly. Add sessions for trajectory memory, then move to task runs for full workflows. Track events with webhooks and monitor costs in your prepaid wallet. Build browser, desktop, and terminal automation that feels like a human operator. Get your API key at https://coasty.ai/developers and start turning your prototype into production.
Want to see this in action?
View Case Studies