PyAutoGUI scripts are easy to write but fragile. A UI element moves and your script breaks. The /v1/parse endpoint turns an existing PyAutoGUI script into clean, structured actions that a computer use agent can run. You do not need credits for this free endpoint. It reads your Python code and emits a list of actions with coordinates, clicks, and keys, ready to feed into Coasty's vision-based computer use API.
How it works
POST https://coasty.ai/v1/parse with a JSON body containing your PyAutoGUI function and imports. The endpoint compiles your function and returns a list of actions, each with a type, x, y, and duration. These actions are the same format used by /v1/sessions/{id}/predict, so you can run them through a stateful trajectory with a single API call. No extra cost and no need for vision calls at this stage.
curl -X POST https://coasty.ai/v1/parse \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"source": "import pyautogui;\n\ndef click_button():\n pyautogui.click(100, 200)\n pyautogui.press('enter')\n"
}'
Request and response fields
- source (string, required): the Python code that defines the function you want to parse, including imports
- output_type (string, optional, default 'actions'): you can request 'structured' or 'actions'
- actions (array, required in response): a list of action objects with fields: type ('click', 'key', 'type', 'screenshot', 'wait'), x (number), y (number), key (string), text (string), duration (number)
- request_id (string, optional): an identifier for this parse request
Run parsed actions through a computer use agent
- POST /v1/sessions to start a stateful trajectory with cua_version 'v3'
- POST /v1/sessions/{id}/predict with the returned actions list as the prompt
- The agent follows your structured steps and can handle dynamic UI changes because it sees the screen
- You are charged only for agent steps at $0.05 each, not for parsing
POST /v1/parse is free, use it to bootstrap vision-based automation from existing PyAutoGUI scripts.
Why this beats brittle automation
Traditional automation relies on static selectors and XPaths that break when the UI changes. Structured actions from /v1/parse give you a clear sequence of clicks and keys, but the computer use agent still sees the screen, adapts to layout shifts, and corrects path drift automatically. You keep the simplicity of PyAutoGUI but gain robustness from vision, not brittle selectors.
Use the free /v1/parse endpoint to convert your existing PyAutoGUI scripts into structured actions and run them through a computer use agent. Build agents that see the screen and act like humans, not brittle selectors. Get a key at https://coasty.ai/developers.
Want to see this in action?
View Case Studies