Most UI automation tools rely on brittle selectors. They break when frameworks update or layouts shift. The Coasty /v1/ground endpoint solves this by grounding UI elements to exact pixel coordinates. You send a screenshot and an element description. Coasty returns x,y for you to use with pyautogui or any other tool. This lets you build computer use agents that see, understand, and act on real interfaces.
How /v1/ground works
The endpoint is POST /v1/ground. It costs $0.03 per call. You provide a base64 screenshot and a text description of the element you want to click. Coasty returns a coordinate pair x and y. This grounding is perfect for pyautogui, robot framework, or any tool that needs pixel values. The process is fast, cheap, and does not require model weights locally.
curl -X POST https://coasty.ai/v1/ground \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"screenshot": "$(base64 -i screen.png | tr -d "\n")",
"element_description": "a green submit button"
}'import base64, os, json, requests
COASTY_API_KEY = os.getenv("COASTY_API_KEY")
with open("screen.png", "rb") as f:
screenshot_b64 = base64.b64encode(f.read()).decode()
resp = requests.post(
"https://coasty.ai/v1/ground",
headers={
"X-API-Key": COASTY_API_KEY,
"Content-Type": "application/json",
},
json={
"screenshot": screenshot_b64,
"element_description": "a green submit button",
},
)
resp.raise_for_status()
coords = resp.json()
print(coords) # => {"x": 512, "y": 340}Grounding vs brittle selectors
- Selectors break on framework updates or layout changes.
- Grounding uses a description and the visual context of the screenshot.
- You get pixel-perfect coordinates you can use with pyautogui.
- Cost is fixed at $0.03 per grounding request.
- No local models or GPU required. Coasty handles the inference.
Use /v1/ground to map any UI description to x,y coordinates for reliable automation.
Where this beats brittle automation
When you rely on selectors, a single class or attribute change breaks your scripts. Grounding does not depend on static selectors. It uses the visual snapshot and a human-readable description. This makes your agents resilient to framework updates, layout shifts, and dynamic content. You can build computer use agents that work on browsers, desktop apps, and terminals the same way a human would see and interact with them.
Start grounding UI elements with /v1/ground and build agents that click, type, and navigate with pixel-perfect reliability. Get your API key at https://coasty.ai/developers .
Want to see this in action?
View Case Studies