Most automation relies on brittle selectors like CSS classes or IDs. When UI changes, your scripts break. A computer use agent sees the screen like a human. But it still needs precise coordinates to click or type. The /v1/ground endpoint turns a screenshot plus a UI element description into exact X,Y click positions. This lets you build agents that adapt to layout changes without brittle selectors.
How /v1/ground works
You send a base64 screenshot and a description of the element you want to interact with. The endpoint returns the bounding box of that element as a JSON object. This box gives you exact X,Y coordinates for your agent's next action. The call is a standard HTTP POST to /v1/ground and costs $0.03 per request.
curl https://coasty.ai/v1/ground \ -H "X-API-Key: $COASTY_API_KEY" \ -H "Content-Type: application/json" \ -d '{
"screenshot": "$(base64 -w 0 screenshot.png)",
"element_description": "The primary CTA button in the top right of the hero section"
}'import base64
import os
import requests
COASTY_API_KEY = os.environ["COASTY_API_KEY"]
url = "https://coasty.ai/v1/ground"
with open("screenshot.png", "rb") as f:
screenshot_base64 = base64.b64encode(f.read()).decode()
data = {
"screenshot": screenshot_base64,
"element_description": "The primary CTA button in the top right of the hero section"
}
resp = requests.post(
url,
headers={
"X-API-Key": COASTY_API_KEY,
"Content-Type": "application/json"
},
json=data
)
resp.raise_for_status()
print(resp.json())Response shape
- The endpoint returns a JSON object with a bounding box for the target element.
- Bounding box coordinates are integers for x, y, width, and height.
- Use the top-left corner (x, y) to click or hover over the element.
Grounding gives you exact coordinates in $0.03 per request.
Where this beats brittle automation
With traditional automation, you hardcode selectors such as .btn-submit or #submit-button. When the UI layout shifts, these selectors no longer match, and your scripts fail. A computer use agent grounded by /v1/ground does not care about class names. It sees the screenshot, understands the description, and finds the click coordinates dynamically. This makes your automation resilient to UI changes and reduces maintenance overhead.
Start grounding UI elements with /v1/ground to build computer use agents that adapt to layout changes. Get your API key at https://coasty.ai/developers and start building reliable visual workflows.
Want to see this in action?
View Case Studies