Web forms and checkout pages often change layout, inject CAPTCHAs, or require mouse movements for fraud checks. Traditional automation tools rely on brittle selectors and API integrations that break when a page loads a new version. The Coasty Computer Use API lets you drive a real desktop environment. You send screenshots, natural language instructions, and receive structured actions. The agent sees the screen, understands context, and clicks or types exactly where needed. This guide shows how to build a checkout flow using the /v1/predict endpoint and the /v1/runs system for long-running tasks.
How it works
The Computer Use API works by repeatedly capturing the current state of a desktop, sending it to a vision model, and executing the returned actions. The key flow uses two endpoints. First, you POST /v1/predict with a base64 screenshot, a text instruction, and the cua_version you want (v3 or v4). The endpoint returns an actions object with coordinates, clicks, and typing steps plus a status field that tells you when the task is done. For multi-step checkout flows you store the session state and loop capture, predict, act until status is done. You can also use the /v1/runs endpoint to let the server drive an agent for you. The /v1/runs endpoint accepts machine_id, task text, optional instructions, system_prompt, max_steps, deadline_seconds, on_awaiting_human (pause/fail/cancel), and a webhook_url. The agent bills $0.05 per step. You can stream events from /v1/runs/{id}/events to track progress.
curl -X POST https://coasty.ai/v1/predict \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"screenshot": "BASE64_SCREENSHOT",
"instruction": "Fill the email field with [email protected] and click the continue button",
"cua_version": "v3"
}'Building a checkout flow
- Start with a stable session using POST /v1/sessions to create a stateful trajectory memory ID.
- Loop: capture screenshot → POST /v1/sessions/{id}/predict → parse actions → execute via pyautogui or similar.
- Use on_awaiting_human to decide what to do when CAPTCHAs or human approval appear (pause/fail/cancel).
- Set deadline_seconds so the checkout flow does not run forever.
- For multi-step tasks you can use POST /v1/runs instead of polling predict, which bills $0.05 per step.
- The server can stream events from /v1/runs/{id}/events with Last-Event-ID for reconnect safety.
POST /v1/predict costs $0.05. POST /v1/runs bills $0.05 per agent step. Use /v1/sessions for stateful trajectory memory.
Where this beats brittle automation
Traditional automation tools depend on CSS selectors, XPath, or API mocks that break when a page loads a new version or injects a CAPTCHA. The Computer Use API agents see the actual rendered UI, read text labels and placeholder attributes, and infer correct targets from context. They can handle dynamically generated IDs, CAPTCHAs, mouse movement checks, and even local fraud prompts. You do not need to maintain a separate scraper for every checkout page. You write a natural language instruction once and the agent adapts to layout changes automatically.
You can now build reliable checkout bots that see the screen and click like a human. Use the Computer Use API to automate forms, handle CAPTCHAs, and run full checkout flows over HTTP. Get your API key and start building at https://coasty.ai/developers.
Want to see this in action?
View Case Studies