OMINX

OMNIX is a local-first, agentic automation architecture designed to execute complex, multi-step workflows entirely on-device within the Android ecosystem. At its core, the project leverages a localized reasoning engine—specifically utilizing the capabilities of the Gemma 4 framework—structured within a strict ReAct (Reason + Act) loop to process user intents and dynamically navigate application interfaces without relying on cloud computation. The key architectural pillars of the system include: Multimodal Sensing & Self-Healing: A dual-path system that prioritizes high-speed Accessibility Tree parsing but seamlessly falls back to a localized visual diffing mechanism (using SSIM math and Vision Transformers) when UI states change unexpectedly or become opaque. This enables the agent to instantly re-learn and self-heal broken navigation paths. Persistent State Management: The system maintains a stateful execution history and episodic memory using local storage, allowing the agent to survive app crashes, resume interrupted tasks, and maintain context across long-running operations. Extensible Plugin Architecture: Moving away from hardcoded prompts, the system utilizes a Model Context Protocol (MCP) approach. Tool schemas (JSON) and domain-specific skills (Markdown) are dynamically injected into the model's context window. This supports highly scalable tool usage and offline "Dream Mode" exploration. Deterministic Safety Guardrails: A strict Human-in-the-Loop (HITL) interceptor sits between the AI's reasoning and the native execution layer. It intercepts the agent's structured JSON tool calls and pauses the automation loop for physical user confirmation whenever sensitive actions (e.g., payments, account deletions) are attempted.

Team

Placement

Participant