Skip to Main Content

Empu

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Empu — Demo video

Empu turns video of a master craftsperson working and explaining into a dictionary you can watch. Every term links back to the moment the master said it, a short clip of their hands doing it, and the actual tool, cropped from the original frame. you can check the app in https://34-71-21-165.sslip.io . The problem is easy to miss. A master batik maker knows the name of every part of a canting, every stage of the process, and local words that change from one village to the next. Almost none of it is written down, and it fades with each generation. Recording a master is easy now, since everyone has a phone. Turning hours of footage into something a student can search is the slow part, and generic speech recognition breaks on craft vocabulary. YouTube’s auto-captions heard “spangkring” as “spang King” and “canting klowong” as “cacing cowok.” So most recordings just sit there as raw files. Here’s how Empu handles it. A documenter uploads a recording, and nothing gets processed until they confirm the contributor’s consent and choose who can see the results (public, community, or private). The video is split into 75-second segments. Muse Spark 1.3 watches and listens to each one, then proposes terms with timestamps, a meaning in English and Indonesian, a confidence level, and a label for how the term was captured: spoken, written on screen, or only seen. SAM 3.1 finds the tool in the frame. After that, a person reviews every proposal in a keyboard-driven studio. Nothing reaches the public dictionary until someone accepts it. We measured whether the model really listens. On our test clip, Muse Spark caught 7 of 8 spoken craft terms. With the audio removed, it caught none. A music-only control clip produced zero invented terms. The Ask panel follows the same rule, answering only from accepted entries, pointing to the clip each claim stands on, and saying it doesn’t know when the archive doesn’t. Craft knowledge lives in a YAML craft pack, not in the code. Batik tulis came first and now has 192 validated entries. Gamelan runs on the same engine with its own pack and 72 terms. Weaving is next. Archives export to JSON, CSV, ELAN for linguists, or a full ZIP, with creator attribution and license on every entry. The demo uses CC BY videos, credited on every entry.

Team