QA testers click buttons, fill forms, and check error messages by eye. For years scripts have tried to mimic this but break when UI changes layout or uses dynamic classes. The Coasty computer use API lets you drive real desktops and browsers visually. You send a task, a screenshot, and an instruction, and the agent returns the actions it would take. This means your QA bot can work against any app without brittle selectors or APIs that do not expose endpoints. In this post you will build a self-running QA bot that opens a web app, fills out a sample form, checks for success, and reports the result.
How it works
Coasty uses a vision model to see the screen and a policy model to decide actions. The agent steps through a sequence of clicks, types, and waits until the task completes or hits a limit. You can run tasks in two ways. The simple way is a single POST /v1/predict call that takes a screenshot, an instruction, and a cua_version. The API returns a list of actions and a status. You loop capture, predict, act until the status is done. For more complex flows with retries and human approval you use POST /v1/runs. This creates a task run with a machine_id, a task description, and options like max_steps and deadline_seconds. The run state tracks queued, running, awaiting_human, succeeded, failed, cancelled, or timed_out. You can stream events with GET /v1/runs/{id}/events to watch progress in real time.
curl https://coasty.ai/v1/runs \
-H "X-API-Key: $COASTY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"machine_id": "machine-123",
"task": "Open the test app, log in, and verify the dashboard is visible.",
"cua_version": "v4",
"max_steps": 150,
"deadline_seconds": 3600,
"on_awaiting_human": "pause"
}'Key configuration options
- machine_id: ID of the cloud VM where the agent runs.
- cua_version: Use 'v4' for autonomous mode with a pass/fail verifier.
- max_steps: Upper bound on agent steps to avoid infinite loops.
- deadline_seconds: How long the agent can work before timing out.
- on_awaiting_human: Choose pause, fail, or cancel when the agent needs approval.
- webhook_url: Optional endpoint to receive run status updates.
Use cua_version v4 for autonomous runs with built-in pass/fail verification.
Where this beats brittle automation
Traditional QA scripts rely on selectors that break when the layout shifts or classes change. The computer use API watches the actual screen, so it works against any UI element even if it has no stable ID or API endpoint. It also handles dynamic content like lazy-loaded lists or modals that appear only after interaction. This means you can test real user flows end-to-end without writing separate scripts for each part of the app. The agent can also handle exceptions such as unexpected popups by using the context it sees. You pay only for the agent steps you use, with a rate of $0.05 per step, so you can scale tests efficiently.
What to build next
Now that you have a self-running QA bot you can expand it into a full test suite. Create reusable workflows that handle login, checkout, and reporting. Integrate the bot into a CI pipeline to run nightly regression tests. For more details and to get your API key go to https://coasty.ai/developers.
Want to see this in action?
View Case Studies