# Model Atlas

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Kaiwen Chen](https://cerebralvalley.ai/u/ckaiwen)
- **GitHub:** https://github.com/ckwflash/model-atlas
- **Demo video:** https://cdn.jsdelivr.net/gh/ckwflash/model-atlas@meta-only-v1/submission/demo.mp4
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4247

Model Atlas is an AI evaluations researcher that uses Meta models to investigate what language models can do, where their performance breaks down, and which conditions deserve a closer look.

Instead of asking only a fixed list of benchmark questions, an investigator model proposes hypotheses, designs fresh experiments, inspects outcomes, and chooses its next tests from the evidence and uncertainty. Every fifth discovery batch explores an unobserved condition so the investigation retains coverage while following interesting leads.

The prototype begins with three independently code-scored task families: compositional string transformations, symbolic state tracking, and constrained extraction. It can also invent new task families and grading rubrics; those model-judged results remain explicitly exploratory.

To test whether adaptation helps, the system compares it with a seeded fixed schedule frozen before any responses, using the same planned discovery allowance. It then freezes nominated hypotheses and generates fresh matched control/stress cases. Exact McNemar tests with Holm correction distinguish confirmed failure patterns from inconclusive results.

The local React/TypeScript workbench connects each finding to the exact prompts, responses, expected answers, settings, seeds, and matched cases behind it. It includes capability maps with uncertainty and unknown regions, hypothesis lineage, pause/resume and restart recovery, usage accounting, recorded replay, and Markdown/JSON exports.

The aim is to make AI-generated evaluations inspectable and reproducible. A negative result is retained rather than turned into a claim of improved performance. Judges can replay the bundled real evidence and a clearly labeled synthetic validation study without an API key.

## More from Meta Global AI Developer Hackathon

- [Team FarmVision](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4244)
- [Leapsify](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4245)
- [SoS](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4246)
- [Timber](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4248)
- [Ehyeh](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4249)
- [Competitor](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4250)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4247. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt