When AI uses the computer for you.

Computer use AI is an agent that operates a computer the way you do. It looks at the screen, then moves the mouse, clicks and types to get a task done. That means it can work in almost any app or website a person can use, even ones with no API or integration.

The trade-off is real. It is slower and less reliable than a direct connection between apps, and anything it reads on screen can try to trick it. As of September 2026, Anthropic, Google, OpenAI and Microsoft all ship a version. Here is how they work, where they help, how well they score and how to use one safely.

01What is computer use AI?

An API is like a staff entrance: fast and direct, but only where someone built one. Computer use walks in through the front door, like any customer. It works almost everywhere, at the cost of speed and certainty.

A computer use agent is still an agent. It runs the same look, plan, act, check loop described in our guide to what an AI agent is. Its tools just happen to be a screen, a mouse and a keyboard.

02How computer use agents see and click

Each turn of the loop is small. The agent gets a view of the screen and picks one action, such as a click at a point or typing a word. The app performs it and returns a fresh view, and the agent decides again. Anthropic’s documentation lists actions like screenshot, click, drag, scroll, type and press a key.

There are two main ways to give an agent its view of the screen:

The accessibility tree is the same structured description of a page that screen readers use. Some tools offer both views. Playwright MCP, an open-source Microsoft tool that lets agents drive a browser, reads the accessibility tree by default and can switch on coordinate-based clicking when needed.

03Where computer use shines

• Apps with no API. Old business systems, supplier portals, government sites and desktop programs.

• Forms and data entry. Copying details from a PDF or spreadsheet into a web form. Microsoft pitches its tool for data entry, invoice processing and data extraction.

• Chores that cross apps. Look something up in one app, paste it into another, save a file, send it for review.

• Web errands. Google aims its auto browse feature at tasks like shopping, travel booking and restaurant reservations.

If a task has an API or a proper integration, use that instead. It will be faster, cheaper and more reliable. Our guide to automating repetitive tasks helps you pick the right tool for each job.

04The limits: speed, reliability and prompt injection

Speed. Every step means a fresh view of the screen and another model call. Anthropic’s documentation puts each screenshot at roughly 1,000 to 1,800 input tokens, so long sessions get slow and costly. On short tasks, a practiced person is often quicker.

Reliability. Clicking is no longer the main problem. The OSWorld 2.0 authors found that agents lose track of constraints, miss information that arrives mid-task, guess rather than ask the user, and skip checking their work. Google’s own help page warns that its agent might misunderstand you and, for example, add the wrong quantity of an item.

Prompt injection. Everything on screen is input, including text an attacker planted. Anthropic’s documentation warns that Claude will sometimes follow commands found in content even when they conflict with your instructions. Our explainer on prompt injection shows how these attacks work.

Defenses are improving. Anthropic reports that on its own browser tests, new safeguards cut attack success against Claude Opus 5 from 3.8% to 0%. But the browser around the agent matters too. In September 2026, researchers published how a malicious extension with two common permissions could hijack the AI assistants in Chrome, Edge, Perplexity Comet, Opera Neon and Claude’s Chrome extension. Chrome and Edge have shipped fixes.

05The main computer use products in 2026

As of September 2026, these are the main ways to put an agent in front of a screen. Benchmarks score the models underneath, not these products.

AI browsers such as Perplexity Comet and Opera Neon take another route and build the assistant into the browser itself.

06How good are computer use agents? The 2026 scores

The benchmark AI labs cite for computer use is OSWorld, which runs agents through real tasks in real desktop apps. In the original version, people completed 72.36% of its 369 tasks, while the best model at launch managed 12.24%.

In June 2026 the OSWorld team released OSWorld 2.0, which is much harder:

Vendors now quote a partial score on OSWorld 2.0, which gives credit for checkpoints reached along the way. These are the published numbers as of September 2026:

The table has gaps. OpenAI’s GPT-6 models, launched this month, had no OSWorld 2.0 score we could verify at the time of writing, and every number above comes from a vendor. For your own work, a ten-minute trial on a real task tells you more than any leaderboard.

07How to use a computer use agent safely

Treat a computer use agent like a temp worker at your desk: give it a clear job, a clean workspace and no access to your wallet. Google’s help page is blunt: you remain responsible for its agent’s actions during a task, “including mistakes.”

A good brief keeps the agent on a short leash:

Task: Download every invoice dated August 2026 from the billing page at portal.example.com/billing. I am already signed in.
Save each PDF to Documents/Invoices/August, named vendor_date_amount.pdf.
Stay on portal.example.com. Do not change settings, pay anything or open other sites.
If a page asks you to sign in, accept terms or confirm a payment, stop and ask me.
When you finish, list the files you saved and anything you skipped.

For the full risk model, including sandboxes and backups, read is it safe to let an AI agent control your computer?

08FAQ

Read next: AI agent vs chatbot, or what MCP is and how agents plug into tools like Playwright MCP.