Ground UI Elements to Coordinates with /v1/ground
Dynamic web apps and desktop tools change their DOM structure and element classes constantly. CSS selectors that worked yesterday break tomorrow. Relying on brittle selectors forces you to rewrite selectors on every minor UI change. The /v1/ground endpoint solves this by grounding element descriptions to exact x,y coordinates on the screen. You give it a screenshot and a description. It returns the pixel-perfect center of the element you want to interact with. This gives you reliable, coordinate-based clicks that survive UI changes. It costs $0.03 per ground call.
How it works
The endpoint accepts a base64 screenshot and an element description. It returns a JSON object with an x and y coordinate. The request uses a base URL of https://coasty.ai/v1 with an Authorization header set to your API key. The payload fields are screenshot (base64) and description (string). The response is an object with x (number) and y (number). You can then use those coordinates with your own automation tool or let the /v1/runs and /v1/sessions endpoints act on the screen.
curl -X POST https://coasty.ai/v1/ground \
-H 'Authorization: Bearer $COASTY_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"screenshot": ""$(base64 -i screenshot.png | tr -d '\n')"",
"description": "search input field"
}'Why ground beats brittle selectors
- ●Selectors break when class names, IDs, or structure change. Ground uses visual descriptions, not selectors.
- ●You can ground elements across different browsers, desktop apps, and hybrid environments with a single prompt.
- ●Ground returns pixel-perfect coordinates. You can combine it with your own automation libraries for full control.
- ●The $0.03 price is lower than many per-step AI agent pricing models, making it cost-effective for bulk grounding tasks.
Grounding UI elements to coordinates with /v1/ground gives you reliable, visual clicks that survive UI changes. Use it after capturing a screenshot and before acting with /v1/runs or your own automation.
Where this beats brittle automation
Traditional automation tools rely on CSS selectors, XPath, or element IDs. These break when a UI framework reorders elements, changes class names, or adds wrapper divs. By grounding to coordinates, you bypass the need to maintain a stable selector tree. You only need a visual description of the element. This lets you automate against any UI that appears on screen, regardless of implementation details. You can also ground multiple elements in a single screenshot for batch planning.
Grounding UI elements to coordinates with /v1/ground is a practical way to make your automation robust against UI changes. Use it to prepare coordinates for your own scripts or feed them into /v1/runs for full computer use automation. Get your API key at https://coasty.ai/developers and start grounding today.