FactoryOps is an AI-powered robotic assembly failure investigation platform. When a robot arm crashes, traditional debugging means manually reviewing hours of sensor logs. FactoryOps eliminates that with two phases: Live Monitoring — A real-time factory floor dashboard synchronizes video, 6-axis force/torque readings, audio waveforms, and 7-DOF joint state data on a shared timeline. Gemini auto-generates operation segment labels and caches incident summaries per recording. AI Investigation — When a crash is detected, FactoryOps dispatches a multi-agent system (Gemini 2.5 Flash). An Orchestrator coordinates four specialists: Vision (sends native video/mp4 clips via ffmpeg directly to Gemini for true temporal analysis), Force-Torque (6-axis spike detection), Audio (acoustic event clustering with structured CANDIDATE_WINDOWS output), and Robot State (trajectory deviation). Agents share a compressed sliding-window memory to prevent context explosion. The orchestrator cross-correlates evidence across all modalities to produce a root cause, confidence score, and recommendations — streamed live to the browser via Vercel AI SDK's Agent UI, with a ReactFlow agent graph animating in real time.
Team
Placement
Participant
