## Definition **Computer use** is an agent modality in which the model operates a computer the way a person does: it **sees** the screen (screenshots, and for the web the DOM / accessibility tree) and **acts** by moving the mouse, clicking, scrolling, and typing. **Browser agents** are the web-specific case. Its point is to reach software that exposes no API. ## How it works A multimodal [[ReAct Pattern|perceive–decide–act loop]] over a GUI: take a screenshot → the model chooses the next UI action (click at these coordinates, type this, scroll) → observe the resulting screen → repeat until the goal is met. ## Why it matters - **Reaches API-less software** — legacy desktop apps, arbitrary websites, anything with only a human interface. - **General** — one interface (the screen) instead of a bespoke integration per application; contrast with structured [[Tool Use]], which needs each capability wired up in advance. ## Risks It is slow, brittle (UIs change and the loop mis-clicks), and higher-risk because it can click *anything* on the machine. It demands sandboxing, [[Prompt Injection]]-aware defenses (a malicious page can hijack the loop), guardrails, and human oversight — see [[Agent Failure Modes]]. ## Related - [[AI Agent]] - [[ReAct Pattern]] - [[Tool Use]] - [[Agent Failure Modes]] ## Sources - Anthropic. *Computer use* (tool-use documentation), 2024–2025.