# Lunatics

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Debjit Jashu](https://cerebralvalley.ai/u/Debjit17)
- **GitHub:** https://github.com/Light172006/issue-to-repro-agent
- **Demo video:** https://www.youtube.com/watch?v=fP-VM9C-nzo
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2549

Issue-to-Repro Agent: from bug report to a verified failing test

The problem. Before a maintainer can fix a bug, someone has to reproduce it. That takes hours, and many reports end in “works on my machine.” A failing test is also the proof that a later fix works.

What it does. Give it a GitHub issue URL. It ends in exactly one of four honest outcomes:

reproduced: a minimal failing pytest test, with a confidence level computed from its checks
needs_info: specific questions drafted for the reporter
not_feasible: the bug needs hardware, an OS or network the sandbox lacks, with the reason
not_reproduced: it tried within limits and could not trigger the symptom

It never posts to GitHub. It only drafts a comment for a human.

How it works. It parses the issue, triages feasibility, then builds the library at the reported version in a locked-down Docker sandbox (non-root, no network, read-only root, resource limits). A tool-using loop on Muse Spark writes and runs tests. Muse Wonder is used only when a run stalls. A deterministic matcher, not the model, decides whether the failure matches the report. A verifier repeats the run 5 times and checks a fixed version. A minimizer then shrinks the test. Every step is logged to a trace, and per-run budget guards cap steps, cost and time.

Evaluation. I built a benchmark of 18 closed bugs from small Python libraries, with a seeded split and a hash-locked held-out set that I never tuned on. I compared the full system against a single-call baseline (A) and a loop without the verifier and minimizer (B).

Held-out (5 feasible, 2 infeasible issues):

Full system: 0/5 reproductions, 1 wrong claim, 2/2 correct refusals
Baseline B: 2/5, 1 wrong claim out of 3, 2/2 refusals
Baseline A: 4/5, but 3 wrong claims out of 7 and 0/2 refusals. It fabricated a test for all 4 infeasible issues across both splits.

The honest result is that the system trades recall for trust. It refuses impossible cases correctly and makes few wrong claims, but on held-out it was too conservative and missed most feasible bugs. The sample is small, so I report raw counts. Dev scores came after tuning and are not evidence of generalization.

Limitations. My minimizer over-shrank tests twice, which made one test fail on the fixed version. Some benchmark entries point to the fixing PR rather than the original issue, so I used only the original issue text in the snapshots.

Built with Python 3.11, Docker, pytest, Typer, httpx, Jinja2, Muse Spark 1.3 and Muse Wonder, during the event.

## More from Meta Global AI Developer Hackathon

- [GFD WEB BRAIN — ARGUS AI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2546)
- [Beucheh Maxime](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2547)
- [Lattice Studios](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2548)
- [Hearth Pantry](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2550)
- [Three Seconds](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2551)
- [QSkill](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2552)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2549. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt