# SpectraPatch

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Edmund Kpodogui](https://cerebralvalley.ai/u/dannkkie)
- **GitHub:** https://github.com/dannkkie/spectrapatch
- **Demo video:** https://youtu.be/sa8hmWubUJA
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5207

SpectraPatch is a hands-free CLI developer agent that detects, patches, and visually verifies temporal mobile and web UI regressions (e.g., animated keyboard occlusions, bottom-sheet bounce collisions, and z-index overlap) that static DOM checkers and accessibility trees falsely mark as passing.

Problem & Impact (30%):
Static test runners inspect widget trees that still report "exists = true" when a critical action button is visually covered by an animated keyboard. Developers spend hours debugging layout shifts across screen ratios. SpectraPatch converts a spoken bug report and a simulator/browser capture into a verified, pull-request-ready code fix with zero manual geometry markup.

Meta AI Multi-Model Architecture (25%):
1. Muse Voice Transcribe: Ingests spoken tester audio via /v1/asr/transcribe in ENDPOINTING mode to extract exact turn-level boundaries (0ms -> 4220ms).
2. Muse Spark 1.3: Extracts normalized colliding UI noun-phrases ("checkout button" vs "keyboard") with strict JSON schema constraints.
3. SAM 3.1 as a Temporal Layout Assertion Engine: Uploads screen recordings once via the Files API and opens parallel streaming sessions per concept across 60 frames (720p normalized). Decodes RLE masks and bounding boxes via meta_sam_parser to compute frame-by-frame pixel collision ratios: Overlap = Area(Mask_A ∩ Mask_B) / Area(Mask_A).
4. Muse Code Sub-Agent (muse exec): Dispatches an isolated git worktree with high reasoning effort to inspect collision keyframes, apply layout fixes (:focus-within safe-area styling and Flutter resizeToAvoidBottomInset), and re-render.
5. Closed-Loop Verification: Headless Chrome re-renders the patched worktree layout; SAM 3.1 re-evaluates the stream to mathematically prove 0.0% overlap before marking the patch verified.

Technical Depth & Provenance (20% & 25%):
- Provenance: Built 100% from scratch during the Meta Global AI Developer Hackathon (Oct 26 – Nov 4, 2026).
- Live Telemetry: Fully operational against production Meta Model API endpoints with live ASR timestamps, real SAM 3.1 video grounding, and worktree git isolation.
- Offline Deterministic Replay: Features a standalone offline verification harness (--demo) backed by 24/24 passing pytest unit/e2e tests so judges can verify the full loop without API credits.

## More from Meta Global AI Developer Hackathon

- [Waveform](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5204)
- [Dayplay AI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5205)
- [OINC Farm Twin](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5206)
- [jimmie](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5208)
- [Madrasati Muse Spark](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5209)
- [AXIS MUNDI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5210)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/5207. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt