Skip to Main Content

BLE Jarvis

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

BLE Jarvis — Demo video

BLE Jarvis: a presence-triggered desk assistant. The phone advertises a BLE service; a Mac menu-bar app derives presence from RSSI (arrive ≥ −60 dBm, hold ≥ −80 dBm, grace window). Presence transitions drive everything, no wake word, no app, no prompt. First arrival of the day: music starts instantly (your Apple Music track via Music.app scripting, with ducking/fades). In parallel, an agent on Muse Spark Open (open weights, served on Baseten) runs a tool-calling loop over calendar, Jira, and news tools and composes a spoken briefing under 150 words. The full loop measures ~10 s — inside the 8–12 s music lead-in, so model latency hides in the intro. Music ducks, ElevenLabs speaks; a window shows the same live data as cards. While you're away, the Mac snapshots your calendar at departure and polls calendar + Jira every 60 s, diffing removals, moves, and additions over a 2-day window. On return: a visual recap plus spoken summary. A canceled meeting becomes a suggestion ("your 15:00 freed up, a swim?"). After each spoken moment a mic window opens; Muse Voice Transcribe handles speech-to-text. "Block it" writes the calendar event; other utterances go to the agent and come back as speech. Learning is plain keep/skip counters, no training, fully inspectable. Goals come from phone onboarding; each activity under a goal has its own record (decline runs twice and it offers swimming, a disliked activity can't sink its goal). Counters are bucketed morning/afternoon/evening; declines stay in their bucket, accepts carry across. Three consecutive declines of one activity trigger a spoken "why?", "not my thing" bans it, "busy week" is unlearned. Twenty minutes into a blocked event, desk presence decides attended vs. bailed and feeds the counters. Silent mode checks the audio route at playback: headphones get voice + music, built-in speakers get the visual-only path. Privacy matches: presence is RSSI, not location; phone–Mac sync is LAN-only behind a pairing code; no cloud, no accounts. The companion app (Expo, iOS + Android) configures everything, goals, voice (with desk-played previews), intro song picked from your Mac's Music library, news feeds. Models: Muse Spark Open (meta-models/muse-spark-open-1026) on Baseten as the default brain, reasoning tokens routed off the content field with an empty-content guard; Muse Spark via the Meta Model API as a config-switchable fallback; Muse Voice Transcribe for STT. Built solo in 8 days, all code written during the hackathon: SwiftUI Mac app, Expo companion app, 171 unit tests (presence hysteresis, calendar diff, learner, sync, audio sequencing). Stated limits: no beacon identity filtering yet; attendance inferred from presence, not GPS; Android background advertising depends on OEM battery policy.

Team