MuseBody

A helper you can just ask for things, like a home robot, needs eyes, ears, a body, and a model that turns “could you bring me that” into an action among real people. Testing that with real hardware is slow and expensive. muse-body tests it in VRChat for free: an avatar that only sees the pixels on screen, hears the audio, and moves with a virtual Xbox gamepad like a player. Muse Voice Transcribe writes down what people say, Muse Spark 1.3 decides what to do from each frame, and SAM 3.1 finds the target again in every frame so a small controller can steer to it. When SAM also matches look-alikes, like menu boards on a wall, Muse Spark picks the real object among numbered boxes. Live, a player asked it out loud for the café menu: it answered, found the menu, grabbed it, handed it over, and told a joke when asked. Every run logs its frames and Meta API calls, published in the repo. Everything was built during the hackathon, including the 3D model of its avatar; the commit history starts Oct 28