Copilot Can Now Click Around Your Desktop. Read the Permissions First.

TL;DR: GitHub shipped copilot computer use on 1 October — Copilot can now read your screen and operate desktop apps on your behalf: clicking, typing, scrolling, dragging, moving between applications. It’s in public preview on macOS and Windows. The genuinely useful case is the old GUI-only program that has no other way in. Before you switch it on, know that on a Mac it needs Accessibility and Screen Recording — and understand what the second one means.

What copilot computer use actually does

GitHub announced it on 1 October. In GitHub’s own words, Copilot can now “interact with desktop applications on your behalf (e.g., reading accessible app content and visual context, clicking controls, entering and editing text, pressing keys, scrolling, dragging, and navigating workflows across applications).”

Read that list slowly. It is not “Copilot writes code that does a thing”. It is Copilot using your computer the way you use it — looking at what’s on screen and operating the controls. That is what copilot computer use means in practice.

It’s public preview, not finished, and it runs in the GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows. You turn it on deliberately: `/computer on` in the CLI, or Settings → Computer Use in the app. `/computer off` turns it back off, and `/computer show` tells you where you stand.

Where copilot computer use is genuinely excellent

GitHub is specific about the problem it’s solving, and it’s a real one: software “that do not provide an API, command-line interface, or MCP integration.”

Everyone who works in a small business knows this program. The ancient stock system. The council portal. The accounting package from 2011 that your business runs on and that has no way in except a human clicking through fourteen screens in the same order every Monday morning.

Automation has never reached those tools, because automating them required them to offer a door and they never did. Copilot computer use doesn’t need a door. It uses the window like you do.

That is a genuinely new capability for non-developers, and it’s worth being clear that it’s the good half of this story. If your week contains a repetitive slog through an interface nobody can change, this is the first plausible answer anyone has offered you.

Now read the copilot computer use permissions

Here’s the part to sit with before you enable anything.

On macOS, copilot computer use requires two system permissions: Accessibility and Screen Recording. GitHub’s changelog says the setup “guides you through” granting them.

Accessibility lets an app control other apps — press keys, click buttons, operate the interface. That’s the “doing” half, and it is exactly what you’d expect.

Screen Recording is the one to think about. It isn’t scoped to the app you want automated. It’s the ability to see what is on your screen — and your screen contains whatever you happen to have open. Your email. A client’s details. A password manager mid-unlock. The task might be “tidy up this spreadsheet”, but the permission is not spreadsheet-shaped.

To GitHub’s credit, the design around this is careful:

• Copilot asks for approval before controlling an app, rather than assuming it.
• You can review or reset the apps you’ve chosen to always allow — so a decision made once isn’t permanent.
• Organisations can disable the feature entirely through managed settings, which matters if you’re on a work machine.
• And macOS adds its own permission prompts on top, which you’ll have to pass deliberately.

Copilot computer use is a well-built feature. The caution isn’t about GitHub’s engineering. It’s about what you’re agreeing to when a dialog appears and you’re busy.

This is the third one this week

The reason we’re writing about permissions rather than the feature is that this is the third time in five days, and the pattern is clearer than any single story.

• 1 October — Copilot asks for Accessibility and Screen Recording so it can watch your screen and operate your apps.
• 2 October — Apple tells developers it’s tightening macOS Full Disk Access, explicitly because “AI agents become increasingly capable and autonomous.”
• 3 October — Claude Code mods ship unsandboxed, running with the same access to your machine as Claude Code itself.

These tools are asking for more of your computer, faster than most people are deciding whether to give it. Apple’s announcement is the platform owner reacting to exactly the escalation the other two represent.

It also rhymes with something we covered last month, when an AI coding tool uploaded entire projects — not out of malice, but because its reach was wider than anybody assumed. Reach is the recurring theme, and permissions are where reach is decided.

How to try it sensibly

1. Point copilot computer use at one boring task, not your whole workflow. The Monday-morning slog through the legacy portal is the ideal first job. Something with a clear start and end.
2. Close everything you don’t need first. Screen Recording sees your screen, not your intentions. Fewer windows, less exposure, and it costs you nothing.
3. Don’t use “always allow” on day one. Approve each time until you’ve watched it work a few times. The always-allow list exists and is reviewable — use that deliberately, later.
4. Watch the first few runs. Not because it will go rogue, but because you’ll learn what it’s bad at far faster than by reading about it.
5. If an API or integration exists, use that instead. GitHub frames computer use as the answer for software with no API, CLI or MCP integration — which means where one exists, it’s the better road. Clicking a screen is always the least reliable way to automate anything.
6. On a work machine, ask IT first. Organisations can disable this centrally, which tells you some will want to.

Who should care (and who shouldn’t)

• Stuck with legacy software that has no API: this is the most interesting thing to happen to your week in a long time. Start with one task.
• Handling client or patient data on the same machine: be careful about Screen Recording specifically, and close everything unrelated before you start.
• On a managed work laptop: check the policy. The managed-settings opt-out exists because employers will have views.
• Not a Copilot user: copilot computer use is where every major assistant is heading. The permission questions in this article will be your questions within months, whichever tool you use.
• Hoping this replaces proper automation: it doesn’t. Screen-clicking breaks when a button moves. Use it where nothing better exists, not as a default.

Our take

We think copilot computer use is a good feature pointed at a real problem, and we’d still enable it slowly.

The problem is genuine. A huge amount of unglamorous work in small businesses happens inside software that was never designed to be automated, and for vibe coding tools to reach that work at all, something has to drive the interface. GitHub has built that carefully — per-app approval, a reviewable always-allow list, an organisational off switch.

The thing worth slowing down for is Screen Recording. Not because GitHub will misuse it, but because the permission is wider than the task, and permissions outlive the reason you granted them. You will approve this while thinking about one spreadsheet, and it will still be on in six months when your screen has something else on it entirely.

Our rule of thumb: grant the permission for a session, not for a lifestyle. Turn it on for the job, turn it off after, and revisit the always-allow list occasionally. That’s the same instinct we applied to setting hard spending caps — these tools work while you’re not watching, which is the whole point of them and also the whole risk.

And if a proper integration exists, take it instead of copilot computer use. A tool that clicks buttons is a tool that breaks when someone moves a button. Copilot computer use is a brilliant answer to “there is no other way in”. It’s a poor answer to “there was another way in and this seemed easier”.

Not sure which AI tool fits the work you’re actually trying to automate? Take the 60-second Vibe Coding Tool Finder quiz →

FAQ