AgencyDiff

Coding agents can ship an update that passes its tests but makes it harder to cancel a subscription, decline location access or block messages from strangers. AgencyDiff checks whether people can still do what they intended after an interface changes. A team defines a product promise as an executable contract. AgencyDiff attempts the same choices on the old and new builds, checks the final app state, and reports changes in effort, visibility and defaults. Every finding includes recorded screens and taps that can be replayed side by side. In an experiment covering six ordinary product tasks, coding agents produced 8 of 30 pull requests with confirmed regressions in users’ ability to say no. With AgencyDiff in the loop, the result was 0 of 30. On a separate benchmark, its replay gate caught 19 of 20 seeded regressions with no false alarms on six unchanged controls. AgencyDiff includes a CLI, GitHub Action, MCP tools and a public report viewer. Its core demonstration uses Circle, a fictional app. The goal is to help teams turn privacy and safety promises into checks they can run before shipping.