Vercel Labs' Agent-Desktop Hits 1.0 With Linux Support for AI Computer Use
The open-source Rust CLI and C library lets AI agents drive desktop apps on macOS, Windows and Linux through accessibility trees instead of screenshots. Version 1.0.2 shipped October 30.
Published
Last updated / Fact-checked on
Vercel Labs' agent-desktop, an open-source tool that lets AI agents read and operate desktop apps, has reached version 1.0. It now runs on Linux as well as macOS and Windows, and version 1.0.2 was published October 30, according to the project's GitHub changelog and npm page.
Photo: Pexels / Pixabay
What it does
The README describes agent-desktop as "a CLI and C library for macOS, Windows and Linux" that gives "any agent reliable computer use on the desktop." It is written in Rust. Instead of guessing from screenshots, it reads each app's real interface structure through the operating system's accessibility tree.
The loop is simple. A snapshot returns JSON with a reference ("ref") for every element an agent can act on. The agent acts on a ref, such as click @s8f3k2p9:e8, then takes a new snapshot to check the result. There is no pixel matching, and the decision-making loop stays in the user's own agent.
Refs are tied to the snapshot that produced them. A bare ref like @e8 is rejected. If the interface changes, an action returns STALE_REF or AMBIGUOUS_TARGET and the agent takes a fresh snapshot. Every command prints one JSON envelope, and errors carry a code and a retry disposition. The README says to retry only when it is marked safe.
Actions are headless by default. They go through the accessibility API without moving the pointer or stealing focus, and a global --headed flag allows physical input.
Release at a glance
| Item | Detail |
|---|---|
| Latest version | 1.0.2 (Oct. 30, 2026) |
| 1.0.0 release | Oct. 27, 2026 (adds Linux via AT-SPI, glibc 2.35+) |
| First npm release | v0.1.2, Feb. 23, 2026 |
| License | Apache-2.0 |
| Commands | 60 names, 56 operational |
| Install | npm install -g agent-desktop (also Bun, Yarn, pnpm) |
| Embedding | libagent_desktop_ffi C library |
What's new in 1.0
The project isn't new. Its first npm release came out in February. Version 1.0.0, dated October 27, added Linux support for the CLI, the C library and the npm package on x86_64 and aarch64 machines running glibc 2.35 or later, using the AT-SPI accessibility layer. It also added verification of release archives, checksums and build provenance.
Version 1.0.1, the same day, removed the project's custom Homebrew tap and Scoop bucket. The README says official Homebrew and Scoop packages are being submitted but are not yet in the default catalogs. Version 1.0.2 fixed macOS permission guidance, restarted the command host after a missing permission is granted, pointed package links at the vercel-labs repository, and added a one-time notice telling npm users when the Node launcher is slowing each call.
Platforms and limits
- macOS needs Accessibility permission (
agent-desktop permissions --guide). The CLI reuses a private command host that exits after 15 idle minutes.TMPDIRmust be owned by the user with mode 0700. - Windows needs no grant for apps at the terminal's integrity level.
- Linux needs a graphical session with
at-spi2-core. Mouse input requires X11 and--headed, and the Wayland clipboard needswl-clipboard.
Of the 60 command names, four (key-down, key-up, mouse-down, mouse-up) refuse to run until a daemon can release a held key or button.
Privacy note
The README warns that snapshots, clipboard reads, screenshots, command output and debug artifacts "can contain private desktop content" and should be kept out of public logs and uploads. An optional JEV helper sends screen descriptions to a configured model provider. It is separate from the native CLI.
Why it matters
Pixel-based computer use breaks when themes, fonts or layouts change. Accessibility-tree refs give agents stable targets and explicit retry rules. Adding Linux means one workflow can now cover all three major desktop operating systems. The tool also teaches agents how to use itself: agent-desktop skills get desktop prints usage rules matched to the installed version.
Not every task can be automated yet. CNBC recently reported that some agent startups still route phone calls and bookings to humans.
More coverage is on our Artificial Intelligence hub.
How we reported this
We read the project's GitHub README, its full CHANGELOG.md and its npm package page. The original draft said Vercel Labs had just open-sourced the tool. The npm version history shows public releases since February 2026, so we reframed the story around the 1.0 milestone. We did not install or test the software.
Published and fact-checked October 31, 2026.
Sources
- GitHub, vercel-labs/agent-desktop: https://github.com/vercel-labs/agent-desktop
- GitHub, agent-desktop CHANGELOG.md: https://github.com/vercel-labs/agent-desktop/blob/main/CHANGELOG.md
- npm, agent-desktop package (v1.0.2): https://www.npmjs.com/package/agent-desktop
DailyMech used AI tools to help research and draft this article; it was fact-checked by our editors.