BeeJEPA

Continuous 60 fps video monitoring of honeybee hive entrances produces high-volume video streams dominated by routine traffic, while critical events unfold in brief, localized bursts on the landing board. Streaming raw 60 fps video directly into cloud vision-language models is computationally and financially impractical, while conventional motion detectors fail in dense insect swarms. beeJEPA, live at https://beejepa.finn-hertsch.de with dataset artifacts published on Hugging Face at F1nnSBK/bee-jepa-entrance-anomalies, solves this with a two-stage Active Perception architecture on Java 25 and GraalVM Native Image: System 1 builds upon a pre-existing university coursework dataset in which a frozen V-JEPA 2.1 ViT-B/16 encoder processes 60 fps side-by-side hive entrance recordings over a 43x43 spatial token grid. Across 96 thirty-second sequences and 2,688 sliding windows, it tracks frame-to-frame token-delta scores and board token peaks against a 99th-percentile threshold to isolate 184 peak anomaly windows. System 2, engineered from scratch during the hackathon, turns those anomaly windows into a grounded inspection console: - A pixel-measured hotspot extractor decodes exact keyframes via ffmpeg, isolates the highest-energy 5x3 token window on the landing board above the ambient noise floor, and renders a 688x688 overview alongside a 384x384 bicubic hotspot crop. - A Java 25 and Micronaut 4.7 runtime stores up to 64 keyframes outside the JVM heap via the Foreign Function and Memory API and evaluates 64-float measured board-energy vectors with the SIMD Vector API to build saccadic trajectories. - Meta SAM 3.1 segments individual bees on the landing board, and our SamGeometry engine intersects those bounding boxes with the measured V-JEPA hotspot using Intersection-over-Area containment at IoA >= 0.25. - Meta Muse Spark Open synthesizes single-event or cohort-wide visual digests and can invoke QuerySam31Concept tool calls on clean keyframes to test open-vocabulary predator or threat concepts before emitting structured Markdown reports and spoken briefings. - Hands-free operator queries are transcribed via Meta Muse Voice Transcribe 1.0 and paired in a Three.js and Vue 3 console with neural speech synthesis, 22.05 kHz bee acoustic synthesis, and a 1024-point FFT acoustic mirror.