Most desktop automation scripts start with PyAutoGUI. You write mouse moves, clicks, and key presses. That code works for you, but it breaks when UI text changes, elements shift, or layout reflows. You need a way to bridge your scripts and a computer use API that can see the screen and act like a human. The free /v1/parse endpoint does exactly that. It takes a PyAutoGUI function body and returns a structured list of actions a computer use agent can execute with vision and pointer control.
How /v1/parse works
The endpoint accepts a POST request with the PyAutoGUI code and returns a parsed action list. The request body is a JSON object with two fields. The source field contains the PyAutoGUI function body as a string. The cua_version field specifies the computer use API version for the agent. The endpoint is free. The response is a JSON object with an actions array. Each action contains an operation (move_to, click, drag_to, scroll, type_text, press_key, release_key, wait, screenshot, save_screenshot, double_click) and coordinates or parameters for that operation. This output is ready to be fed into a computer use agent loop that captures, predicts, and acts until status is done.
import os
import requests
from base64 import b64encode
API_KEY = os.getenv("COASTY_API_KEY")
BASE_URL = "https://coasty.ai/v1"
pyautogui_script = '''
def move_and_click():
pyautogui.moveTo(500, 300)
pyautogui.click()
pyautogui.moveTo(520, 310)
pyautogui.doubleClick()
'''
def parse_pyautogui(code: str) -> dict:
resp = requests.post(
f"{BASE_URL}/parse",
headers={"X-API-Key": API_KEY},
json={
"source": code,
"cua_version": "v3"
}
)
resp.raise_for_status()
return resp.json()
actions = parse_pyautogui(pyautogui_script)
print("Parsed actions:", actions)
# Output example:
# {
# "actions": [
# {"operation": "move_to", "x": 500, "y": 300},
# {"operation": "click"},
# {"operation": "move_to", "x": 520, "y": 310},
# {"operation": "double_click"}
# ]
# }
Key fields and constraints
- The endpoint is free, so you can iterate without cost.
- The request body requires JSON with keys 'source' (PyAutoGUI function body) and 'cua_version' (e.g., 'v3' or 'v4').
- The response contains an 'actions' array where each action has an 'operation' and optional parameters like x, y, text, key, duration, or file_path.
- The parsed actions match valid computer use operations (move_to, click, drag_to, scroll, type_text, press_key, release_key, wait, screenshot, save_screenshot, double_click).
- Only operations supported by the computer use agent are emitted. Unsupported patterns are either skipped or returned as errors in the response.
POST https://coasty.ai/v1/parse with JSON {"source": "PyAutoGUI code", "cua_version": "v3"} to get a list of structured actions for your computer use agent.
Why this beats brittle automation
Traditional automation relies on selectors, IDs, class names, and XPath. When UI text changes or layout shifts, those selectors break and you must update scripts. The computer use API sees the screen and can locate buttons by appearance, text, and layout. By converting PyAutoGUI scripts into structured actions, you inherit the reliability of your original scripts while giving your agent the ability to adapt to UI changes. The parse endpoint bridges the gap between your existing automation knowledge and a robust, vision-based computer use agent.
Start turning your PyAutoGUI scripts into structured actions. Use the free /v1/parse endpoint to generate a list of operations a computer use agent can execute. Build a reusable agent loop that captures, predicts, and acts until status is done. Get your API key at https://coasty.ai/developers and build a computer use agent that actually sees and acts like a human.
Want to see this in action?
View Case Studies