# sanjeev

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Sanjeev Varma](https://cerebralvalley.ai/u/sanjeevarma)
- **GitHub:** https://github.com/sanjeevexe/envoy
- **Demo video:** https://www.youtube.com/watch?v=EIVoCPk8BME
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4466

Muse has a computer. Envoy gives it a phone.

AI agents can write code, clear an inbox and book a flight, but much of daily life runs through phone apps they can't reach: rides, delivery, shopping, social posting. And an agent that does work a phone starts from scratch every time, reading every screen at full model cost.

Envoy is an MCP server that gives agents a phone. It doesn't replace what an agent already does on the desktop and the web; it adds real Android apps, so the two work together. Any MCP client can connect, and we've run it with Meta's Muse, our own agent on Muse Spark, and Claude Code. It runs on a physical phone, a Mac emulator or an Android emulator on Google Cloud.

It acts, tapping and typing like a person. In our demo, Muse compares live Uber and Lyft fares, then takes a clip from the Mac and, in one request, posts it as an Instagram Reel, a YouTube Short and a TikTok, each with a song from that app's own library.

It learns. One tap turns a finished run into a routine: a small program written by Muse Wonder, with inputs, checks around its steps, and targets found by what they are. Envoy replays it once to prove it works. Shared, it becomes a new tool for every allowed agent. In our evaluation, 13 of 18 read-only real-app workflows replayed correctly with 0 model calls at a median 4.88 s, against the agent's 5 calls and 19.59 s.

It repairs itself. When an app changes, Envoy fixes just the broken step and keeps the fix on trial. On SkyLark, our own test app, it recovered from 50 of 50 injected layout changes. Its routines were right 120 of 120 times at a median 1 model call, where a fixed replay managed 38, and repeats cost about 40 times less.

It keeps improving, and we measure it. On Google's AndroidWorld benchmark, Envoy went from 21 to 57 to 74 of 116 tasks (18% to 63.8%) in under a week, through general agent changes and Muse Wonder on its steps. 63.8% is over twice the benchmark paper's best agent (30.6%) and above every tree-only entry on the public, self-reported leaderboard (best 59.5%) (Envoy adds a screenshot on about 11% of steps). We keep running that loop: study the failures, ship general fixes, re-measure.

You stay in control. Envoy gives agents no approve tool, so anything irreversible an agent starts waits for the person's yes on their iPhone. Planted instructions fooled the agent in 0 of 20 runs.

It also watches notifications, offers help before you ask, runs on a schedule, keeps a sourced memory graph, learns from a demonstration, and takes requests by voice or Siri. It has run tasks in Uber, Lyft, Amazon, DoorDash, Starbucks, Gmail, Maps, YouTube, Instagram and TikTok; apps that block emulators go to a real phone.

It's for anyone whose work lives in phone apps: a creator tapping through three apps for every clip, a small business running on ordering and messaging apps, someone for whom tapping is hard, and any agent platform that wants a phone.

## More from Meta Global AI Developer Hackathon

- [kentaurx](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4463)
- [Lucas_Kim](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4464)
- [Mykro AI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4465)
- [Itemized](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4467)
- [Luh Labs](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4468)
- [Troupe](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4469)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4466. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt