Skip to Main Content

Verify SI

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Verify SI — Demo video

I built verify.si to explore how AI agents can check claims of approval while still getting useful work done. The app is for developers building AI agents and researchers studying their safety. Visitors can step through recorded examples involving shared bookings and payments, compare different instructions and tools, and see whether an agent completed the task or acted on a fake approval. Across five studies covering 4,704 short test runs with two Meta models, an optional tool that checks approval when acting was especially promising. When it was offered, the models completed 623 of 624 legitimate tasks and acted on none of 624 fake approvals. A smaller follow-up explored incomplete approvals and messages generated by another model. The interactive replays, code and data let people inspect the evidence and reproduce the results. The findings apply to these controlled tests and do not guarantee safety in real-world use.

Team