# LifeLens

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Dhruv Patil](https://cerebralvalley.ai/u/dhruv_in_space), [Avinash Mantripragada](https://cerebralvalley.ai/u/avinashmantri), [Shreyasvi Natraj](https://cerebralvalley.ai/u/nsrek), [Yash Ingle](https://cerebralvalley.ai/u/yashhere)
- **GitHub:** https://github.com/nshreyasvi/LifeLens
- **Demo video:** https://www.youtube.com/watch?v=z-LMULhrIYQ
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4857

LifeLens - Giving First-Person Vision a Third-Person Understanding

The Problem

Head-mounted cameras and smart glasses capture what a person sees, but they cannot directly see or understand the body behind the camera. This creates a fundamental limitation for AI systems trying to interpret human movement, physical interactions, exercise technique, performance and everyday activities.

Our Solution

LifeLens bridges this gap by reconstructing the wearer's full body in 3D from a single first-person video. It combines hand tracking (HaWoR), camera tracking (DROID-SLAM), full-body reconstruction (OmniEgoCap) and Meta's MHR rig to recover an articulated 3D representation, including fingers.

We render this reconstructed body alongside the original head-camera frame from the same moment and provide both to Meta Muse Spark. This gives the vision model something traditional first-person video cannot: a simultaneous understanding of the environment and the person moving through it.

What It Enables

One reconstructed body powers four experiences:

- Stage: Body-part-aware lighting that follows performers' movements, with Muse Spark designing dynamic lighting scenes.
- Music: Movement-responsive music that adapts to pace and rhythm, with Muse Spark defining musical styles for different sections.
- Gym: Exercise recognition, repetition counting, depth and tempo measurement using 3D joints, with form feedback for every rep.
- Routine: An activity timeline that captures everyday actions. When footage contains no meaningful activity requiring an experience, Muse Spark recognises this and avoids unnecessary outputs.

Technical Implementation & Validation

Built between October 26 and November 4, 2026, LifeLens has been tested on our own GoPro recordings and is available in the project repository.

- User-provided height establishes real-world body scale.
- GoPro lens metadata determines camera focal length.
- Achieved 102.6 mm body reconstruction error across three OmniEgoDB sequences, compared with 111.7 mm reported in the reference paper across 20 sequences.
- Every Muse Spark interaction is logged with its input prompt and response for transparency and debugging.

Technology & Attribution

LifeLens integrates existing research and technologies: HaWoR, DROID-SLAM, OmniEgoCap, Meta MHR, SMPL-H, MANO and ReccoBeats, alongside Spotify integration. These underlying components are not our original inventions. LifeLens's contribution is the integration, reconstruction pipeline and application experiences built around them.

Our project is released under the MIT License, subject to the respective licences and terms of its dependencies.

## More from Meta Global AI Developer Hackathon

- [Mission Informatics](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4854)
- [Lente](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4855)
- [Funhouse](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4856)
- [CleanQuest](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4858)
- [LENHROS](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4859)
- [Codekrafters](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4860)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4857. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt