# Cooking Maniac

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Luke Lu](https://cerebralvalley.ai/u/lukelu0)
- **GitHub:** https://github.com/hsinnosuketw/personal-cooking-coach
- **Demo video:** https://youtu.be/gPmghA52pSc?si=S7SOuuLDIDJzSK_h
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2329

Personal Coaching Chef is a cooking coach that watches someone cook through a head-mounted camera/smart glasses, remembers which recipe steps they have started, and reminds them of steps they have not started yet, so nothing gets forgotten in a busy kitchen. In future version, it will teach people how to make their dishes more delicious.

How it works: Meta's V-JEPA 2 world model reads the video and predicts the next action about 0.8 s ahead, every second. A checklist memory (not started / in progress / done, with rounds for repeated steps) and an HSMM step tracker turn those predictions into recipe progress and decide when the coach should speak. When a step's evidence is unclear, Llama 3.2 11B Vision looks at a few recent frames and answers "has this step begun?". A Muse Glimmer 30B coach writes where the cook is, what is done, what is left and one useful reminder. A small decision model (Liquid AI d1-2B) picks the next action every second, following the coach and the checklist. A task graph of prerequisites lets the coach avoid reminding steps that nothing needs yet when work can run in parallel.

What we measured (small samples, one run per setting, on two HD-EPIC recordings: Scrambled Eggs, and Bacon and Egg Sandwich): frame checks judged "has it begun?" correctly 89% of the time on the eggs dish, and cut how early the Bacon and Egg step memory was wrong from 87 s early to 6 s late. Reminders before the cook starts a step reached 3 of 5 steps (eggs) and 4 of 6 (bacon and egg), up from 0-1 and 1-3. The main limit is that the probe reads HD-EPIC kitchens far worse (10-14% top-5 action accuracy) than the EPIC-KITCHENS footage it was trained on (48%). All results, code, the 40-stage notebook and a side-by-side demo page are in the repo.

This is a personal project built only from public models and the public EPIC-KITCHENS-100 and HD-EPIC datasets (non-commercial research licences).

## More from Meta Global AI Developer Hackathon

- [Pablo Pereyra](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2326)
- [viwaldorf](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2327)
- [Binary](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2328)
- [Lycan](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2330)
- [Saphene](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2331)
- [TEAM AKASHI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2332)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2329. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt