A common bottleneck in automation is the gap between a quick prototype and a reliable production system. You write a script that clicks the right button, but a UI redesign breaks your X paths or CSS selectors. You spend more time patching selectors than building features. The Coasty computer use API solves this by letting an agent see the screen and act like a human, navigating real UIs without brittle selectors. You can start with a single CLI call and scale to multi-step workflows, task runs, and production deployments.
How Coasty works: capture, predict, act
Coasty runs on a capture-predict-act loop. You send a screenshot plus a natural-language instruction to the /v1/predict endpoint. The model returns a list of actions such as click or type, plus a status. When status is not done, you capture a new screenshot and predict again. This loop continues until the model reports success. For longer-running tasks with memory, you create a session with POST /v1/sessions, then POST /v1/sessions/{id}/predict on each capture. The session holds the trajectory so the model remembers what it has done.
bash
# Install dependencies (Python 3.9+)
# pip install requests pillow
export COASTY_API_KEY=$(cat ~/.coasty-key)
# Capture a screenshot of the active window
# (adjust as needed for your OS)
screenshot=$(python3 -c "
import base64
import subprocess
import sys
# macOS example: Grab the active window
cmd = ['screencapture', '-x', '-t', 'png', '-R$(osascript -e 'tell application \"System Events\" to get the bounds of the front window')']
subprocess.run(cmd, check=True)
with open('/tmp/screen.png', 'rb') as f:
print(base64.b64encode(f.read()).decode('utf-8'))
")
# Call the Coasty endpoint
response=$(curl -s https://coasty.ai/v1/predict \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"screenshot\": \"$screenshot\",
\"instruction\": \"Find the 'Sign In' button and click it\",
\"cua_version\": \"v3\"
}")
echo $responseTask Runs: orchestrate a full agent workflow
For production, you often want a task to run to completion, manage retries, handle pauses, and integrate with your infrastructure. Coasty Task Runs are designed for this. You POST /v1/runs with a machine_id, a task, and optional instructions, system_prompt, max_steps, deadline_seconds, webhook_url, and on_awaiting_human settings. The server launches an agent on the cloud VM and drives it step by step. Each step costs $0.05. You can poll GET /v1/runs/{id} for status, stream events via GET /v1/runs/{id}/events, cancel with POST /v1/runs/{id}/cancel, or resume with POST /v1/runs/{id}/resume. States include queued, running, awaiting_human, succeeded, failed, cancelled, and timed_out.
Workflows: versioned JSON DSL
Coasty Workflows let you define reusable, versioned sequences of steps. You POST /v1/workflows with a JSON DSL containing task, assert, if, loop, parallel, human_approval, retry, succeed, and fail step types. Each task step is billed $0.05. You can run a workflow with POST /v1/workflows/{id}/runs or run an inline workflow with POST /v1/workflows/runs. Variables follow the double-brace patterns {inputs.x} and stepId.field. You can hard guard with budget_cents, max_iterations, and deadline_seconds. This is ideal for multi-step production pipelines like onboarding flows, data entry, or testing suites.
Vision and grounding: click by element description
Sometimes you want to target an element by description rather than coordinates. Coasty supports this with two vision endpoints. POST /v1/ground takes a screenshot and an element description and returns x,y coordinates. This is useful when you want to locate a specific button or text. POST /v1/parse is free and converts pyautogui-style commands into structured actions, making it easier to prototype and then scale. Vision endpoints are priced at $0.03 for ground and $0.05 for predict.
POST /v1/runs is the easiest way to move from prototype to production. Each agent step costs $0.05. Start with a task and instructions, then add retries, deadlines, and webhooks as you scale.
Why computer use beats brittle automation
Traditional automation relies on selectors like X paths, CSS classes, or API endpoints. When UIs change, these selectors break, requiring manual patching. Coasty's computer use agent sees the screen and decides where to click and type based on natural language. It can handle dynamic content, pop-ups, and subtle layout changes without code updates. This makes it ideal for browser automation, desktop workflows, and testing where the layout is not stable. You can combine this with workflows and task runs to create resilient production systems.
You now have a clear path from a quick prototype to a production-ready computer use agent. Start with /v1/predict for simple tasks, then move to /v1/runs for full workflows with retries and webhooks. Use /v1/workflows to version your pipelines and hard guard with budget_cents and deadline_seconds. The Coasty platform also provides a cloud VM via /v1/machines so your agent can drive real desktops and browsers. Ready to build? Get your API key at https://coasty.ai/developers.
Want to see this in action?
View Case Studies