Human-Computer Interaction

When we talk about AI agents that can “use a computer like a human,” most of today’s leaders—Claude, GPT-4o, Seed 1.5—are locked in proprietary vaults. This means the critical details that make them competent in high-stakes desktop workflows—training data, error recovery strategies, evaluation methods—are inaccessible to the wider research and business community. OpenCUA aims to change that, not by chasing hype, but by releasing the entire stack: tools, datasets, models, and benchmarks. ...

The GUI Agent Bottleneck: Stuck in Sparse Feedback Training LLM-based GUI agents to complete digital tasks—such as navigating mobile apps or automating workflows—faces a fundamental limitation: reward sparsity. Traditional reward formulations (Outcome Reward Models, or ORMs) provide feedback only at the end of a trajectory. If the task fails, the agent receives zero signal, regardless of how many useful intermediate steps it took. This severely limits credit assignment and slows learning, especially in environments with long action horizons. ...