DuckyHelper

DuckyHelper is a tutor that lives on your computer. Hold fn and ask a question out loud: it sees the screen you are asking about, explains in speech, and draws directly over the relevant words, chart, or diagram. Follow-up questions stay in context, so learning happens where the material already is. For this hackathon, we built DuckyHelper Open, a Meta-powered edition of the existing DuckyHelper product. The starting codebase supplied the native Mac overlay, screen capture/OCR, and drawing runtime. The hackathon contribution is the Meta Model API integration: Muse Spark 1.3 for the screen-aware tutoring and streamed drawing program, Muse Voice Transcribe for push-to-talk questions, SAM 3.1 for grounding marks on objects in the screenshot, and a native local proxy that keeps the learner's API key in macOS Keychain. The open edition removes the need for a DuckyHelper account or a hosted DuckyHelper inference backend. The engineering goes beyond returning chat text. Spark emits an executable teaching program that coordinates speech and drawing; the app runs it as it streams. SAM refines the model's rough visual targets to the objects being discussed, and the learner can point and ask again. The result is a tutor beside your work, with explanations attached to what you are looking at. Submitted to Startups / Build for Your World, and the Most Viral Project bonus. Product: https://duckyhelper.com