# Unity AI Mapper

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Romain Alibar Grawey](https://cerebralvalley.ai/u/DEX-zha)
- **GitHub:** https://github.com/DEX-zha/META-HACKATHON-2026-UNITYMAPPER
- **Demo video:** https://youtu.be/kZxCXbrVMiw
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2861

UnityMapper runs on two Meta models, both called through the Meta Responses API (api.meta.ai/v1/responses): Muse Spark 1.3 for reasoning and SAM 3.1 for segmentation. They are the only models in the pipeline, and every request goes through a reviewed allow-list (model, fields, image count) before it is sent.

Muse Spark 1.3: the agent's brain
Every decision is a single multimodal call (text plus up to 12 images) with a strict JSON-schema output, so nothing reaches Unity until it has been validated against a schema. There's no hidden tool loop: one reservation, one transmission. We use it for:
- Brief outline: rooms, dimensions, style and functional zones.
- Catalogue review: it reads the project's asset catalogue page by page (20 pages for the demo room) and keeps or drops each prefab with a reason.
- Asset review: it looks at two opposite preview renders of a prefab to find its functional front and how it must be supported (floor, tabletop, shelf, wall).
- Layout and layout repair: zone anchors and rotations, re-planned when the geometry solver rejects a proposal.
- Lighting intent: it reads the brief to choose a reference daylight setup taken from a real scene.
- Visual inspection: it reviews Unity captures together with SAM evidence and native geometry, first zone by zone, then for the whole room.
- Correction and enrichment: it proposes fixes and additions, and writes the honest list of unresolved requirements.
- The project chat interface.

SAM 3.1: the agent's eyes
After each build step, Unity renders interior viewpoints. For each view, SAM 3.1 receives a concept prompt derived from the placed prefabs ("bed", "chest of drawers", "rug", "lamp"…) and returns one-bit masks with confidence scores. The agent:
- compares masks with the bounds Unity projects for each object (IoU) as evidence of presence and placement;
- renders the actual masks as colour overlays in the live inspector and in the demo video (517 segmented views in this run);
- caches results by image hash and concept, and never re-sends an identical request. If a SAM result is missing, it falls back to the original pixels and native geometry instead of guessing.

Cost and reliability
A local ledger reserves budget before every Muse Spark or SAM call, using Meta's published per-token and per-image pricing, and records actual usage afterwards. A call with an unknown outcome is never replayed automatically.

## More from Meta Global AI Developer Hackathon

- [bigjames](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2858)
- [Team Drishtikon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2859)
- [Tejasai](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2860)
- [Solo-Arise](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2862)
- [Solar Proposal Agent](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2863)
- [TakeState](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2864)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2861. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt