You have scripts that click pixels and move the mouse. Those scripts break the moment a UI layout changes. You want to turn your PyAutoGUI code into structured action objects you can feed into a computer use API. The Coasty /v1/parse endpoint does just that. It costs nothing and gives you a JSON array of actions your agent can run on real desktops, browsers, and terminals.
How it works
Send a POST request to https://coasty.ai/v1/parse with your PyAutoGUI code. The endpoint returns a list of structured actions like click, type, key_down, key_up, and move. Each action includes x, y coordinates, text, and modifiers. You can then loop through those actions to drive a computer use agent. The endpoint is free and does not use credits.
#!/bin/bash
COASTY_API_KEY="${COASTY_API_KEY}"
py_code=$(cat <<'EOF'
import pyautogui
pyautogui.moveTo(100, 100)
pyautogui.click(200, 200)
pyautogui.typewrite('hello')
EOF
)
curl -sS https://coasty.ai/v1/parse \
-H "X-API-Key: ${COASTY_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"pyautogui_code": '"$py_code"'
}' | jq .Request and response fields
- pyautogui_code (string): your Python code that uses pyautogui.moveTo, pyautogui.click, pyautogui.typewrite, pyautogui.keyDown, pyautogui.keyUp, pyautogui.hotkey, pyautogui.dragTo, or pyautogui.screenshot.
- actions (array of objects): each object has type (string), x (integer), y (integer), text (string), modifiers (array of strings), and duration (number).
- errors: if the endpoint cannot parse a statement, it returns a JSON error with an error object holding code, message, and request_id.
Use the free /v1/parse endpoint to convert PyAutoGUI to structured actions, then feed those actions into your computer use agent.
Where this beats brittle automation
API tools depend on stable selectors and endpoints. When a button moves or a class changes, your script breaks. A computer use agent that sees the screen and acts like a human does not care about IDs or classes. It reads the layout from a screenshot, finds the element, and clicks. By converting PyAutoGUI to structured actions, you preserve your original intent but let the live environment handle layout changes. You still get deterministic steps from your script, but the agent can adapt to minor UI shifts without rewriting selectors.
You can now turn your legacy PyAutoGUI scripts into structured actions for a computer use agent. Use the free /v1/parse endpoint to get started. Next, integrate your actions with a session-driven agent on https://coasty.ai/docs. Get a key at https://coasty.ai/developers and start building agents that feel like real users.
Want to see this in action?
View Case Studies