Pocket Muse

A $55 open-hardware board and a one-day build turn Meta's Muse into a physical companion you talk to - no phone screen required. The problem AI agents live behind glass. For people who can't comfortably read small screens or type - including me - a chat window is the wrong interface. Voice is the interface. But today's voice assistants are trapped in phones and smart speakers that answer trivia; they aren't agents that do things. Meta's Muse Charm announcement pointed at the answer: a tiny, personal, speakable device. I wanted that experience now, on hardware I own and can open up. What I built A working, pocketable Muse companion on the Waveshare ESP32-S3 1.75-inch round AMOLED board - with a second unit, the AIPI Lite, running the same software: - Push-to-talk conversation. Hold the top button and speak. Muse replies as text on the round screen and out loud through the speaker. - Pictures on the device. Ask for a picture and Muse draws it on the 466x466 display. - Real agent work by voice. Weather, questions, facts - the board shares the same live Muse session as the phone app, so the conversation is continuous. - It roams with you. At home it joins house Wi-Fi; walk out the door and it hops to the phone hotspot - verified mid-conversation from the sidewalk. - Phone-free. After a one-time pairing, the device holds its own session and works standalone on battery. The hackathon-new work: the Brain The gadget platform above predates the hackathon. What I built during the hackathon (Oct 27-28) is the Brain - a small skill that lets the board answer with a fact and a picture it drew itself, live: 1. brain.py asks Muse Spark 1.3 (via the Meta Model API) for a one-line fact about a subject plus an image prompt. 2. Muse Image draws the picture (about $0.01 per image). 3. The picture lands on the board's round screen via the display command. The same Brain also runs on Muse Spark Open, served by Baseten - Meta's open weights on a partner platform - proving the skill is not tied to one API lane. How it works - Hardware: Waveshare ESP32-S3 board (466x466 round AMOLED, mic, speaker, battery) - about $55. Second unit: AIPI Lite, about $38. - Firmware: Meta's open-source muse-gadget-sdk, built with ESP-IDF and flashed over USB. One-time pairing in the Muse app; the session token lives on-device and survives reflashing. - Voice-out mod (pre-existing): stock firmware renders replies as text captions only. I wired the SDK's documented start_tts hook to a text-to-speech API: reply text to MP3 to on-device decode to speaker. - Images: the firmware's display command downloads a public image URL and draws it (baseline JPEG, auto-scaled, centered for the round screen). Chat attachments never reach the device - this command is the bridge. - Networking: the firmware stores multiple Wi-Fi networks; the home/hotspot hop is the demo moment that surprises everyone. Why it matters - Accessibility: a voice-first agent for people for whom screens are the barrier - spoken answers and a screen you glance at, not read. - Open and cheap: about $55 of commodity hardware plus open-source firmware. Anyone can build one this weekend. - A glimpse of the post-phone agent: an AI that lives in an object, present in the room - not an app you open. What's next - A wake word inside a small battery's budget (push-to-talk is a deliberate design choice today). - On-device approval for purchases and actions - the Charm's fingerprint idea, done in the open. - Support for newer boards and a printable case. Demo video (final cut, 2 minutes 10 seconds) 1. Intro, indoors - Saint introduces Pocket Muse; the board says hello. 2. Brain run on the Terminal - Spark 200, Image 200. 3. Baseten lane - Spark Open, 200. 4. Outdoor take over the iPhone hotspot - the portrait puffin on the round screen, then a spoken puffin Q&A with the board. 5. Closing line. 6. The public GitHub repo on screen.