# Triton Kernel Harness

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Jishnu Sanyal](https://cerebralvalley.ai/u/Jishy2006)
- **GitHub:** https://github.com/JishyFishy2006/triton-kernel-agent.git
- **Demo video:** https://youtu.be/G3OJjci-lZw
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2671

Triton Kernel Agent: an AI agent that writes fast GPU kernels, and a harness it can't fool.

  LLMs can write GPU kernels, but the speedups they report are easy to fake. Published kernel agents have been caught caching outputs, calling back into PyTorch, quietly dropping to
  lower precision, or timing only part of the work. We built an agent, and a judge built on the assumption that every kernel is trying to cheat.

  How it works: given a PyTorch op from KernelBench, Muse Wonder (Meta Model API) writes a Triton kernel to replace it. Each attempt runs in a fresh Modal H100 sandbox with no
  network and no secrets. Inside it, a trusted parent process never imports the candidate. The kernel runs as an unprivileged user, and only raw tensors pass between the two. The
  parent checks the kernel against an fp64 reference on the visible shape, two input sets and at least five hidden shapes (odd sizes, size one, very large). It times the kernel on
  its own clock, swapping inputs between batches and re-checking outputs after timing. A static scan blocks PyTorch fallbacks, TF32 and fp16/bf16, introspection and hardcoded data. A
  runtime check flags any time faster than the H100's memory bandwidth allows. The result goes back to the model as feedback, with up to 10 attempts per task.

  We attacked our own harness first. A red team of agents wrote 37 attack kernels; the first version fell to 26 of them, so we rebuilt it with process isolation. With the static scan
  turned off, the runtime checks alone now stop 32 of the 37, and every attack is kept as a regression test.

  Results: we ran 43 tasks on an H100 in strict fp32 (12 dev, 31 held out), in three model setups:
  - Wonder only: solved 43/43; held-out geomean 1.28x faster than PyTorch eager.
  - Spark only: 42/43 and 1.01x, at about a third of the cost ($0.37 vs $0.85 per correct task).
  - Wonder writing with Spark fixing compile errors: no better than Wonder alone.

  On single ops the agent is 1.5 to 1.8x faster than eager; on fused GEMM chains in strict fp32, cuBLAS mostly still wins. On softmax and LayerNorm it matches hand-tuned kernels from
  the Triton tutorials. Highlights: a LayerNorm that spreads 16 long rows across all 132 SMs (27x eager, 1.5x torch.compile), and a GEMM-plus-sum task the model rewrote as a column
  sum and dot product (23x).

  The whole batch cost $73 on the API plus about 4 H100-hours, and no runtime cheat check fired. Every attempt, its code and its result are stored in DuckDB and published as a static
  results page.

  Built with: Meta Model API (Muse Wonder 1.0, Muse Spark 1.3), Triton, PyTorch, Modal, DuckDB.

## More from Meta Global AI Developer Hackathon

- [Sprint Serve](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2668)
- [Netag](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2669)
- [Portia](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2670)
- [Sensora AI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2672)
- [BacktestAI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2673)
- [AnyBody](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2674)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2671. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt