# Omni

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Michael Rusinek](https://cerebralvalley.ai/u/michael_rusinek)
- **GitHub:** https://github.com/Mr444-dev/OmniTwin
- **Demo video:** https://youtu.be/Hw48FCp-BRM
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4371

OmniTwin is a real-time Physical-to-3D Digital Twin and verifiable autonomous swarm command system built for the "Push the Frontier" track.

THE PROBLEM:
Testing autonomous robotics, warehouse AGVs, and emergency response swarms requires expensive physical arenas, while deploying raw LLM agents directly into physical control loops is dangerous: LLMs hallucinate coordinates, violate spatial constraints, and introduce multi-second latency that breaks 60+ FPS physics.

OUR SOLUTION:
OmniTwin turns any physical tabletop or workspace viewed through a standard camera into a calibrated 20m x 12.5m 3D Digital Twin in Unity 6 URP, orchestrating 5 autonomous units (3 ground NavMesh AGVs and 2 aerial UAVs equipped with 48-beam, 20 Hz LiDAR sensors).

To bridge cloud AI with hard real-time robotics, OmniTwin implements a Neuro-Symbolic Dual-Loop Architecture:
1. System 1 (Fast Reactive Loop - 200+ FPS in Unity 6 & .NET 10): Converts 2D camera observations via 4-point projective homography, OpenCV Sparse Optical Flow, and a Hungarian assignment tracker into live 3D obstacle proxies with dynamic NavMesh carving and local 48-ray LiDAR collision braking.
2. System 2 (Verifiable Cognitive Brain - Muse Spark Open + Roslyn + Windows AppContainer): Instead of asking an LLM to output raw motor ticks, our Brain uses Muse Spark Open to synthesize C# strategy classes implementing ISwarmPolicy. Every generated C# policy is statically verified via Roslyn SemanticModel symbol whitelisting (with a 3-attempt compiler-diagnostic self-healing loop) and executed inside a zero-capability Windows AppContainer + Job Object OS sandbox (no filesystem, network, or credential access, 100 ms compute deadline, ~0.32 ms IPC round-trip). If synthesis or runtime validation fails, the system deterministically falls back to verified policies (Patrol, Hazard Cordon, Grid Search, or Emergency Hold).

BUILT 100% DURING THE HACKATHON:
- Full .NET 10 & .NET Standard 2.1 architecture backed by 438/438 passing automated tests.
- Sustained 230–480+ FPS at native 1920x1080 (Direct3D 12) with 0 MB texture memory growth.

## More from Meta Global AI Developer Hackathon

- [Bienfait Aksanti](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4368)
- [Second Eyes](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4369)
- [Atom Industries](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4370)
- [Merit](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4372)
- [EigenBrains](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4373)
- [erdos](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4374)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4371. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt