Let your agent see your screen, click buttons, type text, and automate desktop workflows.
Computer use lets your AI agent interact with your Mac like a human would — taking screenshots, seeing what's on screen, clicking buttons, and typing text.
This unlocks powerful automation workflows:
Your agent doesn't just click coordinates — it sees and understands what's on screen using Claude's vision capabilities. It adapts to UI changes automatically.
When you ask your agent to interact with the desktop, here's what happens:
This loop repeats until the task is complete. Your agent narrates each step so you know what's happening.
To enable computer use, Pinchr needs two macOS permissions:
Lets Pinchr capture screenshots. Everything stays local — screenshots are never uploaded unless you explicitly share them.
Lets Pinchr control mouse and keyboard input to click buttons and type text.
If you skipped these during onboarding, you can grant them anytime:
You can revoke these permissions anytime from System Settings → Privacy & Security. Your agent will notify you if it needs these permissions to complete a task.
Here are some examples of what you can automate with computer use:
"Fill out this expense report with data from my receipts folder."
Your agent opens the form, reads receipt files, enters amounts, dates, categories, and submits.
"Test the checkout flow in our web app and verify the confirmation page appears."
Your agent navigates the UI, adds items to cart, completes checkout, and verifies success visually.
"Check our analytics dashboard every hour and alert me if traffic drops below 1000 visitors."
Your agent takes a screenshot, reads the metrics visually, and sends you a Slack alert if needed.
"Walk me through setting up a new Slack workspace."
Your agent highlights UI elements, explains each step, and can even perform actions for you.
"Export all customer records from the old CRM to CSV."
Your agent navigates the legacy app UI, clicks through menus, exports data — no API required.
Pinchr includes safeguards to prevent unintended actions:
Watch the first time. When trying computer use for the first time, keep an eye on what your agent is doing. Once you're comfortable, you can let it run unattended.
Computer use is powerful but has some limitations:
For tasks with API access, use MCP servers instead — they're faster and more reliable. Computer use shines when no API exists or visual verification is needed.
You can record computer use workflows and replay them as scripts:
Scripts are stored in ~/.pinchr/workflows/ as JSON. You can edit them manually for fine-tuning.
Join our community or reach out — we're here to help.