Training computer-use agents needs thousands of realistic interactions. Real desktop sessions are hard to collect, risky to expose, and expensive to label. Synthetic trajectories let you generate high-fidelity interaction data at scale without the overhead of live environments.
Why real trajectories are a bottleneck
Most companies struggle to get a few thousand labeled desktop steps. A typical workflow for an AI assistant might require 10,000+ labeled interactions to reach stable performance. Manual labeling is slow and expensive, often costing $30, $50 per hour. Even then, the resulting datasets are narrow in scope and often include privacy-sensitive information. Synthetic data lets you fill that gap with data that matches your target domain but removes sensitive details and reduces labeling costs.
What a trajectory actually is
A trajectory is the full sequence of actions an agent takes to complete a goal. It includes mouse movements, clicks, keystrokes, browser history, DOM state snapshots, and final outputs. For training, you need both the instruction and the full execution path. For evaluation, you compare the agent's predicted trajectory against the ground truth. Synthetic data lets you generate these sequences in bulk while preserving the statistical properties of real user behavior.
Key tradeoffs to consider
- Coverage vs. fidelity: Synthetic environments can simulate thousands of tasks but may miss rare edge cases found only in production.
- Domain fit: If your agents work with proprietary tools, synthetic data must be built from real interactions in those tools to capture the right UI patterns.
- Labeling cost: Synthetic data can reduce labeling by 50, 80% compared to manual annotation, but you still need human verification for critical tasks.
- Privacy and compliance: Synthetic data removes personally identifiable information and can comply with data residency and export rules.
The most effective approach is to start with synthetic data for breadth and scale, then validate and refine with a small set of real interactions.
How Coasty fits
Coasty runs computer-use agents on real desktops and browsers to capture realistic interaction data. This enables the creation of custom synthetic datasets and trajectories that reflect actual workflow patterns. Coasty delivers synthetic data as a custom, contact-led service tailored to your use case. There is no self-serve product or fixed package, you work with the Coasty team to define the data requirements and scope.
If you need synthetic trajectories for agent training or evaluation, the best next step is to talk to the Coasty data team. Book a data call to explore what customized synthetic data can do for your project at https://cal.com/coasty/coasty-data-call .
Want to see this in action?
View Case Studies