Samuel is a free, open-source voice AI desktop assistant for macOS. It watches your screen, listens to system audio, browses the web with a real Chromium browser via Playwright, and writes its own plugins at runtime using GPT-5.5 with reasoning. When a plugin fails, Samuel diagnoses and repairs it automatically. Use cases include language learning, hands-free email and browser control, live meeting interpretation, and general desktop automation. Released under the MIT license. Built by Sam Feng.
Meet Samuel
An AI that lives with you.
And grows with you.
Other AIs sit behind a tab. Samuel lives on your Mac — quiet, attentive, and a little curious. He sees what you see, hears what you hear, and the moment you ask for something he can't do yet, he simply learns how.
Not an app. A presence.
You don't open Samuel. You just talk to him.
No menus. No tabs. No copy-pasting your screen into a chat box. He's already there — watching what you're working on, listening for the moment you need him.
He sees what you see.
Watching anime, reading a paper, debugging a recipe — Samuel quietly takes in the screen so you never have to explain context.
He hears what you hear.
The Japanese news on your screen, the lecture in your headphones, the meeting in another tab — he listens alongside you.
He answers in your voice's pause.
No typing, no buttons. Speak the way you'd speak to a friend sitting beside you. Samuel replies in roughly half a second.
The part nobody else does
Ask for anything.
He'll build the ability if it doesn't exist.
Most AI products are limited to whatever the company shipped. Samuel isn't. When you ask for something new — a weather widget, a stock tracker, a way to read your calendar out loud — he writes the tool, tests it, and installs it. While you're still talking.
And when something he built breaks later? He fixes it himself, then casually mentions what happened.
Where he lives, what he can do
A new shape for AI.
We didn't make a smarter chatbot. We made something with a different silhouette — closer to a roommate than a tool.
The potential
The ceiling isn't what we shipped.
It's what you'll ask for.
Every conversation teaches Samuel a little more about how you work. Every new ability he builds becomes part of him. The longer you live with him, the more your Samuel he becomes — quieter where you want quiet, sharper where you want sharp, fluent in the things you care about.
Questions, plainly answered
FAQ
- What is Samuel?
- Samuel is a free, open-source voice AI desktop assistant for macOS. He watches your screen, listens to system audio, browses the web with a real browser, and writes his own tools on demand using GPT-5.5. He runs locally on your Mac and is released under the MIT license.
- How is Samuel different from ChatGPT, Siri, or Alexa?
- ChatGPT lives in a browser tab and only sees what you paste. Siri and Alexa run scripted commands. Samuel lives on your Mac and continuously sees your screen, hears your audio, and — uniquely — writes brand-new tools for himself when you ask for something he can't yet do. He also auto-repairs his own tools when they break.
- What can I use Samuel for?
- Common uses: language learning while watching content (anime, foreign news, lectures), hands-free web browsing ("show me my emails", "check my GitHub"), live meeting interpretation, building custom AI tools by voice, song teaching with lyric correction, and general desktop automation.
- Does Samuel really write his own tools?
- Yes. When you ask for something Samuel doesn't already do — for example, a weather widget — he generates the code with GPT-5.5, has it reviewed by GPT-4o-mini, validates it, and installs it without restarting. If the new tool breaks later, he diagnoses the failure and patches it automatically (up to two attempts before asking for help).
- How does Samuel browse the web?
- Three ways. (1) Quick search via SerpAPI. (2) Deep research via the OpenAI Responses API with web search. (3) For sites that need a login — Gmail, GitHub, your bank — Samuel opens a real, visible Chromium browser via Playwright, you sign in once, and he reads and interacts with the page like a human.
- Which operating systems does Samuel support?
- macOS 14 (Sonoma) or newer, only. Windows and Linux ports are on the public roadmap.
- Is Samuel free?
- The Samuel application is free and open source under the MIT license. You pay OpenAI directly for API usage. Typical voice conversation runs at standard Realtime API pricing; ambient screen and audio observation runs roughly $0.02–0.05 per minute while active.
- Is my data private?
-
Memory, preferences, skills, plugins, and API keys are stored
locally on your Mac in
~/.samuel/. Browser sessions run locally via Playwright. Screen captures and audio are sent to OpenAI for processing only while a feature is active, and you can toggle screen watching and audio listening off at any time. - What models does Samuel use?
- OpenAI Realtime API for voice conversation, GPT-5.5 with reasoning for plugin code generation and repair, GPT-4o Vision for screen understanding, GPT-4o-mini for code review, and Whisper for transcription.
- How do I install Samuel?
- A one-click installer is in progress. For now, install requires building from source on macOS using Node.js, Rust, and Tauri. The complete instructions are in the GitHub repository. Join the early-access list above and you'll be notified the moment the packaged installer ships.
Early access
Be one of the first
to live with him.
Open source. Mac only, for now. The packaged installer is on the way — drop your email and we'll let you know the moment it's ready.
Or peek at the source — github.com/sambuild04/screen-voice-agent