Most computer-use agents start by grabbing screenshots and asking the model to click. The simplest pattern is a loop: capture, predict, act, repeat. That works, but every new screenshot resets the context. The model has to relearn which window is open and where the mouse just moved. Coasty offers two ways to handle that: a stateless predict loop with /v1/predict and a stateful sessions flow with /v1/sessions and /v1/sessions/{id}/predict. The sessions flow adds trajectory memory so the model remembers previous steps, clicks, and text. This lets you build longer workflows without re-scanning the screen every turn.
How it works
Use /v1/predict for short, independent tasks. The request needs a base64 screenshot, an instruction, and cua_version. The response includes actions and a status. If status is not done, capture again and call /v1/predict with the new screenshot and the same instruction. This is a stateless loop that charges $0.05 per predict call. For longer workflows, create a stateful session with POST /v1/sessions. The response gives you a session_id. Then POST /v1/sessions/{id}/predict with the screenshot, the instruction, and cua_version. The response again includes actions and a status. The key difference is that the server stores the trajectory, clicks, inputs, and perceived screen changes, so the next predict call can reference the previous state. This costs $0.04 per predict call inside a session. You can also call /v1/ground to map a screenshot plus an element description to an x,y coordinate for coarse clicks, billed $0.03. Both flows can call POST /v1/runs to let the server drive an agent to completion. A /v1/runs request needs a machine_id, a task, and cua_version (default v3). It supports optional instructions, system_prompt, max_steps, deadline_seconds, on_awaiting_human, and webhook_url. Each agent step costs $0.05. You can cancel or resume a run with POST /v1/runs/{id}/cancel and POST /v1/runs/{id}/resume. GET /v1/runs and GET /v1/runs/{id} list or inspect runs, while GET /v1/runs/{id}/events streams Server-Sent Events with reconnect support.
Example: Stateless predict loop
export COASTY_API_KEY
# Capture a screenshot (replace with your own screenshot capture command)
SCREENSHOT=$(base64 -i screenshot.png | tr -d '\n')
CURL_OPTS="-s -X POST -H 'X-API-Key: $COASTY_API_KEY' -H 'Content-Type: application/json'"
STATUS="pending"
INSTRUCTION="Click the 'Save' button in the top right corner."
while [ "$STATUS" != "done" ]; do
RESPONSE=$(curl $CURL_OPTS -d '{
"screenshot": "'$SCREENSHOT'",
"instruction": "'$INSTRUCTION'",
"cua_version": "v3"
}' https://coasty.ai/v1/predict)
echo "$RESPONSE" | jq -r '.actions.status'
STATUS=$(echo "$RESPONSE" | jq -r '.status')
if [ "$STATUS" != "done" ]; then
sleep 1
# In real code, capture the next screenshot and store it in SCREENSHOT
fi
done
# Billed: $0.05 per /v1/predict callWhen to choose one over the other
- Use /v1/predict when tasks are short, isolated, or one-off. The model sees a fresh image each turn, which can be simpler for very short actions like a single click or a quick form field entry.
- Use /v1/sessions for multi-step workflows where the model must remember context over several turns. Trajectory memory lets the model refer to previous clicks, inputs, and perceived state without relearning.
- Each predict call inside a session costs $0.04, while the stateless predict is $0.05. For longer workflows, sessions can be cheaper and more accurate.
- POST /v1/runs lets you offload the loop and keep-alive to the server. It supports max_steps, deadline_seconds, and webhook callbacks. Each agent step costs $0.05.
- POST /v1/ground maps a screenshot plus an element description to coordinates. Use this for coarse clicks before fine-tuning actions with predict.
- POST /v1/parse converts pyautogui code into structured actions for reuse.
Use /v1/sessions for multi-step workflows and /v1/predict for short, independent actions.
Where this beats brittle automation
Traditional tools rely on selectors, IDs, and XPath that break when layouts change. A computer-use agent looks at the screen, understands the current state, and can move the mouse or type naturally. Stateful sessions let it remember where it clicked and what it saw, so it can adapt to small UI shifts without failing. Stateless predict is still powerful for one-off clicks where you do not need history. The key is picking the flow that matches the complexity of your task and the stability of your UI.
Start with a stateless predict loop for simple clicks, then move to /v1/sessions when you need longer workflows. If you want the server to handle keep-alive and error recovery, try POST /v1/runs. Either way, you are building a computer-use agent that behaves like a human, not a brittle script. Get your API key at https://coasty.ai/developers and start driving real desktops, browsers, and terminals.
Want to see this in action?
View Case Studies