Computer-use AI agents: what they can and cannot do yet

Computer-use AI agents are AI models that operate a computer screen. You give one a goal, it looks at a screenshot, and it returns an action such as moving the cursor, clicking a button, typing text or scrolling. The software carries out that action, a fresh screenshot is taken, and the loop repeats until the task is finished or the agent stops.

They matter because some software can only be used by clicking through its screens, so a person has to do that work by hand. An agent that could do it reliably could take on some of that work. The same ability also makes these agents a target. A web page can contain instructions that the agent mistakes for the user's own request.

What has been demonstrated

Most of the solid evidence comes from benchmarks, which are standard sets of tasks used to compare systems. The results below come from research groups running published benchmarks. Most are preprints, which have not been peer reviewed, and each result holds only for the models and settings tested.

WebArena, a 2023 preprint, provides functional websites from four domains: e-commerce, social forum discussions, collaborative software development and content management. Its best GPT-4-based agent completed 14.41% of tasks from start to finish, against a human performance rate of 78.24% (WebArena). These are 2023 results and do not describe today's systems.

OSWorld, an arXiv preprint submitted in April 2024, provides 369 computer tasks in real web and desktop applications. The best model tested, GPT-4 given a text description of the screen's interface elements (its accessibility tree), completed 12.24% of them, against a human baseline of 72.36% (OSWorld). The authors say the models struggled mainly with GUI grounding, meaning the step of matching a described element, such as "the Save button", to the right place on the screen, and with operational knowledge.

UI-CUBE, a November 2025 preprint from the automation company UiPath, has 226 tasks. Its authors tested five agents: Claude Computer Use 4.0, OpenAI's computer-use-preview, and UiPath's own agent running GPT-5, GPT-5 mini or Gemini 2.5 Flash. Simple interface interactions succeeded 67% to 85% of the time across these agents, against 97.9% for humans. On complex workflows the agents reached 9% to 19%, while human testers with no prior experience of the applications reached 61.2% (UI-CUBE).

Longer tasks are harder still. OSWorld 2.0, a preprint first posted in June 2026 and revised in July 2026, contains 108 long-horizon workflows that take human users a median of about 1.6 hours. The best system in that preprint, Claude Opus 4.8 run with maximum thinking and batched tool calls, completed 20.6% of tasks under a strict pass or fail measure with a limit of 500 steps (OSWorld 2.0). This was the top score in the preprint's July 2026 version; newer models may score differently.

Results also vary from run to run. An April 2026 preprint that ran OSWorld tasks repeatedly points to three sources of variation: randomness during execution, ambiguity in how tasks are described, and differences in agent behaviour. It does not give a single headline success rate (On the Reliability of Computer Use Agents).

What is reported but not independently checked

Much of the more recent progress is described in company announcements or benchmark maintainers' posts. These are claims, not independent measurements.

Anthropic introduced computer use in public beta in October 2024. Its announcement said the upgraded Claude 3.5 Sonnet scored 14.9% on the screenshot-only OSWorld category, rising to 22.0% when given more steps, and that the next-best system scored 7.8% (Anthropic). The same post called the feature "still experimental" and "at times cumbersome and error-prone", and said that some actions "currently present challenges for Claude".

OSWorld-Verified, a revised version of the benchmark released by its maintainers in July 2025, reported CoACT-1, an agent framework, as the leader at 60.76% (XLANG Lab). The post does not name the model behind CoACT-1 or state the step budget for that score, and it notes that it ran models under different step settings, with one model's score ranging from 9.1% to 23.0% depending on the budget. This was the leading score in July 2025, not today.

The security problem

In 2023, Greshake and colleagues, in an arXiv preprint, showed that an attacker can plant instructions in data that an AI application is likely to retrieve, and so compromise the application without direct access to it (Greshake et al.). That paper is cited for the type of attack, not for a success rate.

Anthropic has published its own figures on browser agents. For its Chrome browser extension, in August 2025, it reported attack success of 23.6% without safety mitigations and 11.2% with mitigations in autonomous mode, across 123 test cases in 29 attack scenarios (Anthropic). In November 2025 it reported an attack success rate of about 1% for Claude Opus 4.5 with improved safeguards, against its own internal adaptive attacker given 100 attempts per environment (its chart shows 1.4%). That post covers browser use rather than full computer use. It calls a 1% rate "a significant improvement" that "still represents meaningful risk", and says that prompt injection is "far from a solved problem" (Anthropic). The two sets of figures use different test setups, so they cannot be compared directly.

Limits and open problems

• Long tasks. Scores are much lower on long, multi-step workflows than on simple interactions, as the UI-CUBE and OSWorld 2.0 results show.
• Benchmarks versus everyday use. Benchmark tasks are set and controlled. The sources reviewed for this article include no independent field study of agents doing ordinary office or web work without close supervision, so it is not known what share of real tasks they would complete.
• Prompt injection. Because the agent reads untrusted content, an attack can try to redirect it. The reported defences lower the rate, but the developer itself says the remaining risk is meaningful.
• Consistency. The same task can produce different results on different runs, so a single success rate may overstate how dependable an agent is.
• Cost and speed. The sources reviewed do not cover running costs or how quickly agents work, so this article does not assess them.

Where it could help (speculative)

Several uses are proposed but not yet shown. One is automating routine tasks in older business software that has no programming interface. Another is testing software by using it the way an ordinary user would, to find faults before release. A third is helping people who find a mouse or keyboard hard to use by carrying out multi-step tasks from a spoken or written goal. None of these has been tested at scale in the sources reviewed.

What to watch next

• Independent results on OSWorld 2.0 or OSWorld-Verified for a system in general release, with the test setup published.
• Repeated-run results showing success rates across many attempts at the same tasks.
• A field study in which agents work on office or web tasks with human checks removed for a set period.
• Prompt injection rates for full computer use, not only browser use, with the test setup published.
• Whether agents that ask for approval before purchases, submissions or account changes stay the default in vendor products.