Skip to Main Content

VisionPilot

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

VisionPilot — Demo video

VisionPilot uses a web browser the way a person does. You type a task in plain English, like "change the alert threshold to 80% and save." It takes a screenshot, puts a numbered box on every button, and asks Meta's Muse Spark model for one next step, like "type 80 into number 3." Before anything that saves or deletes, it stops and asks me. After every action it takes a second screenshot and Muse Spark checks whether the step really worked. It repeats this until the screen shows the task is done. I tested it on three tasks on a practice admin site I built: reading the errors on a dashboard, changing a setting and saving it, and the same task with a surprise pop-up. In the pop-up run, the checker said "an unexpected modal popup appeared," and VisionPilot's next step was to close it. That moment is the whole point of the project: it doesn't just follow a plan, it notices when something changes.

Team