Turn PyAutoGUI to Structured Actions with the Free Parse Endpoint
You have a PyAutoGUI script that automates a repetitive task. It uses sleep, click, and press. That script is brittle. If the UI changes, the coordinates break. You need something that understands what is on the screen and can adapt. The Coasty computer use API gives you that with vision and actions. But starting from a raw PyAutoGUI script still requires you to hand-translate each line into a structured action. The parse endpoint solves that problem. It takes a PyAutoGUI script as input and outputs a structured action list you can run directly in a task run or a workflow. It is free, so you can experiment without spending credits.
How the parse endpoint works
The parse endpoint transforms a PyAutoGUI Python script into a structured list of actions. You POST the raw Python code to /v1/parse with the body containing a code field. The endpoint returns a list of actions where each action has an action_type and a params object. The params hold the specific arguments for that action, such as x, y, button, key, or text. This structured output is what the Coasty computer use agent expects when it receives instructions via the predict endpoint or when you run a task run. The parse endpoint does not execute the code for you. It only produces a representation that can be executed later. It is free, so you can parse as many small scripts as you need while you build your automation DSL.
curl -X POST https://coasty.ai/v1/parse \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"code": "import pyautogui\npyautogui.moveTo(100, 200, duration=0.5)\npyautogui.click()\npyautogui.write("hello", interval=0.1)\npyautogui.press("enter")"
}'What the response looks like
- ●The parse endpoint returns a JSON object with a single top-level field: actions.
- ●Each action in the actions array has an action_type such as click, move, write, press, or scroll.
- ●The params object contains the exact parameters needed to execute that action, for example { "x": 100, "y": 200, "button": "left" } for a move and click.
- ●You can iterate over the actions array and feed them into a task run or a workflow step that expects structured actions.
- ●Because the endpoint is free, you can parse multiple scripts and compare their outputs before committing credits to a full computer use agent run.
POST /v1/parse with a code field and read the actions array from the JSON response.
Where this beats brittle automation
PyAutoGUI scripts rely on hardcoded coordinates. If a window moves or the layout changes, the automation breaks. The parse endpoint does not fix that yet, but it gives you a structured representation that is independent of the original PyAutoGUI call stack. Once you have a structured action list, you can combine it with vision-based computer use. For instance, you can let the agent scan the screen, locate an element by description using the /v1/ground endpoint, or use the /v1/predict endpoint to decide where to click next. This hybrid approach keeps the speed of a script while adding the adaptability of a computer use agent. You avoid brittle selectors because the agent can reason about the visual context, and you avoid the overhead of rewriting every line from scratch.
Start by taking your existing PyAutoGUI scripts, feeding them to the parse endpoint, and inspecting the generated actions. Use that structured output as a foundation for your own automation DSL. Once you are comfortable, plug those actions into a task run or workflow to let the Coasty computer use agent execute them on real desktops or browsers. Get your API key at https://coasty.ai/developers and begin turning scripts into scalable, vision-aware automation.