Building robust UI automation with pure API calls is brittle. You need to know every selector in advance. You need to keep APIs stable. The /v1/predict endpoint fixes that. It sends you a screenshot, a natural language instruction, and a CUA version, then returns a list of actions. The agent sees what you see and plans the next click, type, or scroll. This is the core of a computer use agent.
How /v1/predict works
The endpoint POST https://coasty.ai/v1/predict expects a base64 screenshot, an instruction, and a cua_version. It costs $0.05. You send a screenshot of your target UI, describe what you want to do, and tell Coasty which CUA version to use. The server returns a JSON response with a status field and an actions array. Each action includes type (click, type, scroll, etc.), x, y coordinates, text, or other relevant fields. The status field can be done, pending, or error. When status is done, the actions array contains all steps to complete the task. You then capture a new screenshot and loop predict until done.
curl https://coasty.ai/v1/predict \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"screenshot": "iVBORw0KGgoAAAANSUhEUgAA...",
"instruction": "Click the orange submit button",
"cua_version": "v3"
}'Request and response fields
- screenshot: base64 encoded image (PNG or JPEG).
- instruction: natural language description of the goal.
- cua_version: version of the computer use agent (default v3).
- status: done, pending, or error.
- actions: array of action objects. Each action has type, x, y, text, and any additional fields required for that action type.
- price: $0.05 per predict call.
Loop capture, predict, and apply actions until status is done.
Where this beats brittle automation
With pure selectors you need to know every CSS class or XPath in advance. If a UI changes, your script breaks. With /v1/predict the agent receives a screenshot and an instruction, so it can react to visual changes. It can click buttons that move, recover from layout shifts, and handle dynamic content. This makes your computer use agent resilient to minor UI changes without breaking. It also enables workflows that require visual understanding, such as reading error messages or following visual flows, which are hard with API-only tools.
Use /v1/predict to build a computer use agent that sees and acts like a human. Start by capturing a screenshot, calling predict, and applying the returned actions. Loop until the status is done. Once you are comfortable, explore stateful sessions with /v1/sessions or run full task workflows with /v1/runs. Get your API key at https://coasty.ai/developers and start automating with computer use.
Want to see this in action?
View Case Studies