# Hack Singh

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Sidak Sethi](https://cerebralvalley.ai/u/Sidak)
- **GitHub:** https://github.com/SidakSethi-Singh/agentproof
- **Demo video:** https://drive.google.com/file/d/1XaE_Qv-5ItGU0fNpSoyEpF9X_dJDXKSU/view?usp=sharing
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2580

AgentProof is an automated multilingual agent-evaluation harness designed to measure capability gaps, tool-calling consistency, and run-to-run variance across Indic languages (English, Hindi, and Punjabi).

THE PROBLEM IT SOLVES:
AI agents are overwhelmingly evaluated on English-only benchmarks. When models are deployed in multilingual environments, tool invocation degrades, date normalization fails, and run-to-run variance spikes. Without standardized multilingual evaluation harnesses, these agentic reliability gaps remain completely invisible to developers.

HOW IT WORKS:
1. Standardized Benchmark: Tests the agent across 6 tasks (arithmetic planning and temporal lookup) across English, Hindi, and Punjabi, repeated 3 times each (18 runs total).
2. Dual-Scoring Harness: Every run is evaluated by both a deterministic Python code checker (normalizing numbers and multilingual scripts) and an LLM rubric judge with structured JSON output enforcing correctness, tool adherence, and language fidelity.
3. Real Empirical Findings: In our evaluation of Meta Llama 3.2 3B, English scored 50% (3/6), Hindi scored 0% (0/6), and Punjabi scored 50% (3/6). The harness captured genuine failure modes: on the English calculator, the agent hallucinated an extra ×2 factor (answering 42 km/h instead of 84 km/h); on the Hindi calendar, it failed to normalize Devanagari month strings, leading to a tool error and a hallucinated day (Saturday instead of Sunday); while on the Punjabi calculator, it passed 3/3 with 84 km/h.

DELIVERABLES:
- Interactive Streamlit dashboard with KPI metrics, variance charts, and full tool-call trace inspector.
- Automated Model Card (MODEL_CARD.md) following Meta conventions and raw CSV data export.

## More from Meta Global AI Developer Hackathon

- [Tether](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2577)
- [Team Zekraa](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2578)
- [Team Estrella](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2579)
- [George (GK) — TRUE IP / FME, Inc.](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2581)
- [Thalamuse](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2582)
- [Vulcan](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2583)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2580. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt