remy

My grandmother has late-stage Parkinson’s. Every time she stands up to go somewhere on her own, falling is close to a coin toss, and what worries my family most is her going down and not being able to call for help. Right now our safety net is taking turns checking a live camera feed from her room. That leaves way too much to chance. remy is an agent that watches a normal home camera and handles a fall the way a family member would. When someone goes down, it confirms it’s actually a fall, asks “Are you okay?” through the camera’s own speaker, listens to the answer (or the silence), and messages the family on WhatsApp. If she says she’s fine, the family gets a calm message with what she said. If she doesn’t answer, it asks once more, then sends an urgent alert with a clip of the fall and tells her help is on the way. If she calls for help, it skips the second question and escalates right away. And if she just sat down on the couch, nobody gets pinged. A small YOLO pose model runs locally all the time and watches for a fast drop. When it sees one, remy cuts a 5 second clip and sends it to SAM 3.1, which tracks the person. A rule on that track (how far the body dropped, whether it ends up lying down, whether they get back up) decides if it was a fall. Muse Spark looks at the clip and writes the note the family reads. Then the camera asks “Are you okay?” and records 15 seconds, Muse Voice Transcribe turns the reply into text, and Muse Spark decides whether it means okay, needs help, unclear or no answer. Words like “help me” skip straight to urgent. There’s also a floor rule, so lying still on the floor for 20 seconds raises an event even if the drop wasn’t seen. The Meta models only get called when something actually happens, which keeps it at about 2 cents per fall. I tested it on 100 clips from public fall datasets and it caught 32 of 40 falls with 1 false alarm in 60 non-falls. With thresholds picked on held-out halves it still catches 80%. The time to confirm a fall went from 23 seconds in my first version to about 5, by running SAM while YOLO keeps watching for a get-up instead of waiting a fixed amount of time, and it asks “Are you okay?” about 11 seconds after a fall. On the real camera in my living room, all four staged cases (okay, no answer, help, and sitting on the couch) ended the way they should. It runs on a $20 Tapo C200 from Amazon and a laptop, and all of it was built from scratch during the hackathon (first commit Oct 29). The repo has a keyless replay of a recorded event and the eval scripts, so you can check the numbers without a camera or an API key. Next up is better hardware, a fully local setup so video never leaves the house, and real-world testing in my grandmother’s room.