Omni

OmniTwin is a real-time Physical-to-3D Digital Twin and verifiable autonomous swarm command system built for the "Push the Frontier" track. THE PROBLEM: Testing autonomous robotics, warehouse AGVs, and emergency response swarms requires expensive physical arenas, while deploying raw LLM agents directly into physical control loops is dangerous: LLMs hallucinate coordinates, violate spatial constraints, and introduce multi-second latency that breaks 60+ FPS physics. OUR SOLUTION: OmniTwin turns any physical tabletop or workspace viewed through a standard camera into a calibrated 20m x 12.5m 3D Digital Twin in Unity 6 URP, orchestrating 5 autonomous units (3 ground NavMesh AGVs and 2 aerial UAVs equipped with 48-beam, 20 Hz LiDAR sensors). To bridge cloud AI with hard real-time robotics, OmniTwin implements a Neuro-Symbolic Dual-Loop Architecture: 1. System 1 (Fast Reactive Loop - 200+ FPS in Unity 6 & .NET 10): Converts 2D camera observations via 4-point projective homography, OpenCV Sparse Optical Flow, and a Hungarian assignment tracker into live 3D obstacle proxies with dynamic NavMesh carving and local 48-ray LiDAR collision braking. 2. System 2 (Verifiable Cognitive Brain - Muse Spark Open + Roslyn + Windows AppContainer): Instead of asking an LLM to output raw motor ticks, our Brain uses Muse Spark Open to synthesize C# strategy classes implementing ISwarmPolicy. Every generated C# policy is statically verified via Roslyn SemanticModel symbol whitelisting (with a 3-attempt compiler-diagnostic self-healing loop) and executed inside a zero-capability Windows AppContainer + Job Object OS sandbox (no filesystem, network, or credential access, 100 ms compute deadline, ~0.32 ms IPC round-trip). If synthesis or runtime validation fails, the system deterministically falls back to verified policies (Patrol, Hazard Cordon, Grid Search, or Emergency Hold). BUILT 100% DURING THE HACKATHON: - Full .NET 10 & .NET Standard 2.1 architecture backed by 438/438 passing automated tests. - Sustained 230–480+ FPS at native 1920x1080 (Direct3D 12) with 0 MB texture memory growth.