# Pocket Muse

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Sin Chan](https://cerebralvalley.ai/u/SaintSinC)
- **GitHub:** https://github.com/SaintSinC/pocket-muse
- **Demo video:** https://youtu.be/MRb7OAK5X2s
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/912

A $55 open-hardware board and a one-day build turn Meta's Muse into a physical companion you talk to - no phone screen required.

The problem

AI agents live behind glass. For people who can't comfortably read small screens or type - including me - a chat window is the wrong interface. Voice is the interface. But today's voice assistants are trapped in phones and smart speakers that answer trivia; they aren't agents that do things.

Meta's Muse Charm announcement pointed at the answer: a tiny, personal, speakable device. I wanted that experience now, on hardware I own and can open up.

What I built

A working, pocketable Muse companion on the Waveshare ESP32-S3 1.75-inch round AMOLED board - with a second unit, the AIPI Lite, running the same software:

- Push-to-talk conversation. Hold the top button and speak. Muse replies as text on the round screen and out loud through the speaker.
- Pictures on the device. Ask for a picture and Muse draws it on the 466x466 display.
- Real agent work by voice. Weather, questions, facts - the board shares the same live Muse session as the phone app, so the conversation is continuous.
- It roams with you. At home it joins house Wi-Fi; walk out the door and it hops to the phone hotspot - verified mid-conversation from the sidewalk.
- Phone-free. After a one-time pairing, the device holds its own session and works standalone on battery.

The hackathon-new work: the Brain

The gadget platform above predates the hackathon. What I built during the hackathon (Oct 27-28) is the Brain - a small skill that lets the board answer with a fact and a picture it drew itself, live:

1. brain.py asks Muse Spark 1.3 (via the Meta Model API) for a one-line fact about a subject plus an image prompt.
2. Muse Image draws the picture (about $0.01 per image).
3. The picture lands on the board's round screen via the display command.

The same Brain also runs on Muse Spark Open, served by Baseten - Meta's open weights on a partner platform - proving the skill is not tied to one API lane.

How it works

- Hardware: Waveshare ESP32-S3 board (466x466 round AMOLED, mic, speaker, battery) - about $55. Second unit: AIPI Lite, about $38.
- Firmware: Meta's open-source muse-gadget-sdk, built with ESP-IDF and flashed over USB. One-time pairing in the Muse app; the session token lives on-device and survives reflashing.
- Voice-out mod (pre-existing): stock firmware renders replies as text captions only. I wired the SDK's documented start_tts hook to a text-to-speech API: reply text to MP3 to on-device decode to speaker.
- Images: the firmware's display command downloads a public image URL and draws it (baseline JPEG, auto-scaled, centered for the round screen). Chat attachments never reach the device - this command is the bridge.
- Networking: the firmware stores multiple Wi-Fi networks; the home/hotspot hop is the demo moment that surprises everyone.

Why it matters

- Accessibility: a voice-first agent for people for whom screens are the barrier - spoken answers and a screen you glance at, not read.
- Open and cheap: about $55 of commodity hardware plus open-source firmware. Anyone can build one this weekend.
- A glimpse of the post-phone agent: an AI that lives in an object, present in the room - not an app you open.

What's next

- A wake word inside a small battery's budget (push-to-talk is a deliberate design choice today).
- On-device approval for purchases and actions - the Charm's fingerprint idea, done in the open.
- Support for newer boards and a printable case.

Demo video (final cut, 2 minutes 10 seconds)

1. Intro, indoors - Saint introduces Pocket Muse; the board says hello.
2. Brain run on the Terminal - Spark 200, Image 200.
3. Baseten lane - Spark Open, 200.
4. Outdoor take over the iPhone hotspot - the portrait puffin on the round screen, then a spoken puffin Q&A with the board.
5. Closing line.
6. The public GitHub repo on screen.

## More from Meta Global AI Developer Hackathon

- [RaceLab](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/909)
- [Outsiderz](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/910)
- [Hearth_MoveIn](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/911)
- [MeowlSolos](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/913)
- [Ronnie Cutamora](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/914)
- [Dynatech](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/915)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/912. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt