Most browser and desktop automations start with pyautogui. You click, type, and scroll by absolute coordinates. This works until the layout changes or the window moves. You end up with fragile tests and maintenance costs. The Coasty Computer Use API lets you go from raw pyautogui to structured, semantic actions. The /v1/parse endpoint does the heavy lifting for free.
How the parse endpoint works
The /v1/parse endpoint accepts a raw pyautogui script and returns a structured list of actions. The request body includes a list of steps. Each step must have a type and optional arguments. Valid types include click, type, scroll, screenshot, wait, and any custom step you define. The response is a JSON object with a list of actions and a status field. Use the actions directly in your computer use agent or workflows.
import os
import base64
import json
import requests
BASE_URL = "https://coasty.ai/v1"
API_KEY = os.getenv("COASTY_API_KEY")
# Your raw pyautogui script as a list of steps
to_parse = [
{
"type": "click",
"x": 500,
"y": 300
},
{
"type": "type",
"text": "Hello, Coasty",
"x": 500,
"y": 320
},
{
"type": "scroll",
"dx": 0,
"dy": 100
}
]
# POST to /v1/parse (free)
url = f"{BASE_URL}/parse"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
resp = requests.post(url, json={"steps": to_parse}, headers=headers)
resp.raise_for_status()
parsed = resp.json()
print(json.dumps(parsed, indent=2))Key options and constraints
- The request must contain a steps array with each step having a type field.
- Optional arguments like x, y, text, dx, dy, and delay are passed through to the structured actions.
- The endpoint is free and returns a JSON object with a list of actions and a status field.
- You can extend the action schema with custom step types for your own workflows.
- Error responses include an error object with code, message, and request_id.
POST https://coasty.ai/v1/parse with a steps array to get structured computer use actions for free.
Why this beats brittle selectors
Selector-based automation relies on CSS classes, IDs, and XPath that break when layouts change. The parse endpoint works on generic pyautogui-like primitives. It lets you maintain a single source of truth in your own scripting language. Later you can consume the structured actions in workflows, sessions, or task runs. This keeps your automation resilient to UI shifts and reduces the number of selectors you must maintain.
Start converting your pyautogui scripts into structured computer use actions today. Use the free /v1/parse endpoint to simplify your automation pipeline. Get a key at https://coasty.ai/developers and build robust agents that drive real desktops and browsers.
Want to see this in action?
View Case Studies