Ground UI Elements to Coordinates with the /v1/ground Computer Use API Endpoint
Building a computer use agent means you have to act on the real UI. Clicking by index or guessing coordinates breaks the moment a layout changes. The /v1/ground endpoint solves that by taking a screenshot plus a natural language description of an element and returning its exact x and y pixel coordinates. This gives your agent the same spatial awareness a human has.
How /v1/ground works
The endpoint is a stateless HTTP call. Send a base64 screenshot, an instruction that describes the target element, and the CUA version (use v4 for the latest behavior). The service analyzes the visual context, locates the element, and returns actions with a status. Grounding gives you precise coordinates for click, type, and drag actions.
curl https://coasty.ai/v1/ground \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"screenshot": "BASE64_SCREENSHOT",
"instruction": "the blue delete button in the top right corner",
"cua_version": "v4"
}'Request fields
- ●screenshot: base64-encoded image of the screen you want to ground on.
- ●instruction: natural language description of the target UI element.
- ●cua_version: version of the computer use agent logic (default v4).
Response fields
- ●actions: array of parsed actions including click, type, etc.
- ●status: current state (e.g., done, processing).
Grounding is billed at $0.03 per call.
Where this beats brittle automation
Traditional automation relies on CSS selectors, XPath, or hardcoded indices that break with layout shifts or theming changes. Grounding lets your agent rely on visual semantics and natural language descriptions instead. It works across browsers, apps, and OS windows without needing stable selectors. This is what makes a computer use agent truly robust.
Start grounding your UI actions to exact coordinates with the /v1/ground endpoint. Build agents that see and click like real users. Get your API key at https://coasty.ai/developers.