# WorldSTATE

- **Event:** [The Harness Engineering & Model Wrangling #SFTechWeek Hackathon](https://cerebralvalley.ai/e/mongodb-sftw-hackathon)
- **When:** Sun, Oct 11 at 9:00 AM – 10:00 PM (PDT)
- **Where:** San Francisco, CA
- **Placement:** Finalist
- **Team:** [Arnav Dewan](https://cerebralvalley.ai/u/arnavdewan), [Thilak Shekhar Shriyan](https://cerebralvalley.ai/u/thilakshriyan)
- **GitHub:** https://github.com/arnxv0/worldstate-mongodb
- **Demo video:** https://youtu.be/y532iVUl6y4
- **Gallery:** https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/18

WORLDSTATE gives physical spaces a memory. Any phone becomes an observer: open a link, point the camera, and WORLDSTATE tracks what's inside fridges, shelves and tables, what changed, and who changed it. Then anyone can just ask: "Hey World, where's the milk?" or "What's in fridge 2?" It answers out loud in about 2 seconds, with when and where it last saw the item.

How it works:

Detection on the device: MediaPipe on the laptop detects fridges and registered objects. Frames only go to the cloud when something relevant is in view.
Perception: the vision model reads only the key frames. It catalogues contents, finds changes ("−milk") and tells whether this is the same fridge or a new one, using registered reference photos ("register this").
Memory: MongoDB Atlas holds the live state, the event history and long-term memory, and streams updates to a shared dashboard in real time.
Voice: ElevenLabs speech-to-text and text-to-speech; answers come from hybrid search over the stored memory.
Use cases: home kitchens and pantries, warehouses, commercial kitchens, shared equipment, and finding misplaced items. Anywhere tracking is manual and coordination costs time.

## More from The Harness Engineering & Model Wrangling #SFTechWeek Hackathon

- [Muscle Memory](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/15)
- [ammas health](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/16)
- [Probity](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/17)
- [Agent46](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/19)
- [MongoMation](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/20)
- [Expunge](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/21)

---

Markdown version of https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/18. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt