ControlAnything

ControlAnything is a Meta Quest 3 passthrough app plus a server on your laptop. Meta's SAM 3.1 outlines real objects. You pinch one and ask what your hands should do. A Muse agent on the laptop (Muse Spark with terminal, file, deploy and test tools) writes a connector, runs it, tests it on the device, fixes failures, and returns AR controls or a hand program. Nothing is pre-wired. It works with any device the laptop can reach: USB, MIDI, serial, Bluetooth or network. Demonstrated: a visual synth you touch like water, Instagram Reels scrolled by hand swipes, an OP-1 Field over USB-MIDI, a DJI gimbal over Bluetooth, and a virtual pet. Every demo can also record head and hand poses for physical-AI datasets. A phone or webcam version needs no headset.