Skip to Main Content

PhysicalMCP

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

PhysicalMCP — Demo video

Physical MCP gives machines with no API an API. Lots of lab and factory machines only have physical buttons and a small screen, so a person has to walk over and press them. Our agent does it for them. A webcam watches the control panel, SAM 3.1 finds the buttons and screens, Muse Spark reads the screen and plans the presses, and a servo finger presses the real buttons. After every press the camera checks the screen and retries if it didn't work. Before anything risky, like START, Muse Spark Open scores the risk and explains why, and the job waits for a human to hold to approve. If the screen can't be read, it pauses instead of pressing blind. In our demo it sets a kitchen timer to 2 hours and starts it. On Nov 3, 7 of the last 8 full jobs on the real rig worked (jobs stopped by hand not counted), and all 23 recorded runs are in the repo. A recorded full job cost about $0.14 in API calls.

Team