Flaky selectors break your automation. A button might move, a class name might change, or the text might be slightly different. You need a way for your computer use agent to find the right spot on the screen without brittle selectors or hardcoded offsets. The /v1/ground endpoint gives you that. It takes a base64 screenshot and a description of what you want to click and returns the exact x, y coordinates. It costs $0.03 per request and works with any screenshot you capture from a browser, desktop app, or terminal.
How it works
You send a POST request to https://coasty.ai/v1/ground with an Authorization header containing your API key. The body should include a base64-encoded screenshot and a text description of the UI element you want to locate. The endpoint processes the image, finds the matching region, and returns the top-left x, y coordinate of that element. You can then pass those coordinates to your agent's action step to click, type, or hover over the correct spot.
curl https://coasty.ai/v1/ground \ -H "Authorization: Bearer $COASTY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"screenshot": "$SCREENSHOT", "description": "the blue submit button with the text sign in"}'import os, base64, requests
api_key = os.getenv("COASTY_API_KEY")
url = "https://coasty.ai/v1/ground"
with open("screenshot.png", "rb") as f:
img_bytes = f.read()
screenshot_b64 = base64.b64encode(img_bytes).decode("utf-8")
resp = requests.post(
url,
json={
"screenshot": screenshot_b64,
"description": "the blue submit button with the text sign in"
},
headers={"Authorization": f"Bearer {api_key}"}
)
resp.raise_for_status()
result = resp.json()
print("x:", result["x"], "y:", result["y"])Endpoint details
- Endpoint: POST /v1/ground
- Price: $0.03 per request
- Required headers: Authorization: Bearer <key>
- Body fields: screenshot (base64 string), description (text)
- Response includes x and y coordinates of the element's top-left corner
Call /v1/ground once per element, then reuse the coordinates in your agent's click action.
Where this beats brittle automation
Static selectors fail when layouts change or when you automate across different environments. The /v1/ground endpoint relies on visual understanding. Your agent describes what it sees, the API finds the matching region, and you get a reliable coordinate. This works in headless browsers, real desktop apps, and mixed terminal environments without maintaining a separate selector database. It fits naturally into a computer use agent workflow where you capture a screenshot, ground the target, and perform the action.
Add /v1/ground to your computer use agent pipeline to make clicks and interactions robust across any UI. Get an API key at https://coasty.ai/developers and start grounding your automation.
Want to see this in action?
View Case Studies