# DPI Research

- **Event:** [PyTorch Helion Hackathon](https://cerebralvalley.ai/e/helion-hackathon)
- **When:** Sat, Mar 14 at 9:00 AM – 10:00 PM (PDT)
- **Where:** Shack15, San Francisco, CA
- **Team:** [Daniel Pang](https://cerebralvalley.ai/u/danp)
- **GitHub:** https://github.com/dpiresearch/Helion_20260314/blob/main/helion/causal_conv1d_py/submission.py
- **Gallery:** https://cerebralvalley.ai/e/helion-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/helion-hackathon/hackathon/gallery/22

Not sure where the f8_quant submission is supposed to go, so here it is: https://github.com/dpiresearch/Helion_20260314/blob/main/helion/fp8_quant_py/submission.py

For causal_conf1d

Tuned the Helion execution config so the kernel uses the GPU more effectively:
num_warps: 1 → 4
More threads per block (4×32 = 128), so more parallelism and better occupancy.
num_stages: 1 → 2
More pipeline stages so load/store and compute can overlap better (better memory latency hiding).

For the gated_deltanet_* kernels

Here’s a concise summary of what the gated_deltanet\* submission kernels improve over the reference implementations.

1. gated_deltanet_chunk_fwd_h

Reference (reference.py): PyTorch eager, sequential over chunks. For each chunk it:
Keeps full chunk tensors k_c, w_c, u_c, g_c in memory
Does v_new_c[:, c] = u_c[:, c] - w_c[:, c] @ h and h = h * exp(g_last) + k_c[:, c].T @ v_gated
Loops over c in Python; lots of intermediates and global memory traffic

Improvements in submission (submission.py):

Single Helion kernel with explicit tiling: tiles over (B*H, V) with block [1, 8] and over time T with chunk size C = 64.
Recurrent state in registers: keeps state [K, V] per (b, h) and updates it chunk-by-chunk with hl.dot for the matmuls (e.g. w @ state, k_adj.T @ diff), avoiding full chunk-sized matmuls in global memory.
Specialized sizes: K and V are hl.specialize(...) so the compiler can optimize for fixed dimensions.
Stable config: One helion.Config (e.g. num_warps=4, num_stages=2) for all shapes to stay within the leaderboard timeout while still being GPU-friendly.
So the main improvement is moving from a chunk-by-chunk Python loop with big PyTorch matmuls to one tiled Helion kernel that keeps the recurrence in small state and uses hl.dot for the core math.

2. gated_deltanet_chunk_fwd_o

Reference: Reshapes to chunks and does:
o_inter = (q_c @ h) * exp(g_c) (inter-chunk)
Full C×C qk = q_c @ k_c.T * exp(g_diff) with causal mask, then o = (o_inter + qk @ v_c) * scale
Large temporary tensors for the full chunk–chunk attention.

Improvements in submission:

Tiled over (BH, T) with block [1, C] so each program works on one chunk of time for one (b, h).
Same math, better mapping:
Intra-chunk: qk = hl.dot(q_tile, k_tile.T), causal mask and g_diff, then sim = where(causal, qk * exp(g_diff), 0) and local_out = hl.dot(sim, v_tile).
Inter-chunk: global_out = hl.dot(q_s, h[c_idx]) with q_s = q_tile * exp(g_vals).
Output: (global_out + local_out) * scale.
Helion dot primitives instead of large PyTorch matmuls, so the work is expressed as smaller, cache-friendly ops.
Config: e.g. num_warps=8, num_stages=4 for more parallelism on this matmul-heavy kernel.
So the improvement is preserving the exact chunk-fwd-o formula while executing it in a tiled, dot-based way that fits the GPU and avoids huge intermediates.

3. gated_deltanet_recompute_w_u

Reference: Reshapes to (B, NT, C, H, K/V) and does two batched matmuls:
u_c = A_c @ (v_c * beta_c)
w_c = A_c @ (k_c * (beta_c * exp(g_c)))
then permute/reshape back to (B, T, H, K/V).

Improvements in submission:

No big batched matmuls: the matmul is expressed as two explicit passes over the inner dimension ci in 0..C-1 and ci in C-1..0, each accumulating:
w_acc += a_col[:, None] * (k_ci * coeff_ci * decay_ci)[None, :]
u_acc += a_col[:, None] * (v_ci * coeff_ci)[None, :]
Averaging: w_out = (w_acc1 + w_acc2) * 0.5, u_out = (u_acc1 + u_acc2) * 0.5. That gives a symmetric (forward+backward) order of summation, which can improve numerical behavior (e.g. cancellation) compared to a single pass.
Tiling: over (B*H, T) with block [1, C], so each tile handles one chunk and the inner loop is over C with small accumulators (hl.zeros([rt, K]), hl.zeros([rt, V])), which can stay in registers or fast memory.
Config: e.g. num_warps=4, num_stages=2 for stability under the 12-minute leaderboard limit.
So the improvement is replacing one-shot batched matmuls with a tiled, two-pass accumulation that is both GPU-friendly and numerically more stable.

Cross-cutting improvements

Static shapes and configs: All three use static_shapes=True and SHAPE_CONFIGS keyed by (B, T, H, K, V) so the right kernel/config is chosen per shape without autotuning on the bot.
Single config for timeout: Comments note that B200-tuned or autotuned configs hit the 12-minute leaderboard timeout; using one “safe” config per kernel improves reliability.
IEEE dot precision: All use dot_precision="ieee" for correctness.
Optional ACF: Comments point to advanced_controls_file for further tuning (e.g. booster pack ACFs) once a baseline is correct and stable.
In short: the gated_deltanet submissions improve over the references by turning chunk-wise PyTorch loops and large matmuls into single, tiled Helion kernels that use small state, explicit dots, and (for recompute_w_u) a two-pass summed form for better numerics and GPU utilization.

---

Markdown version of https://cerebralvalley.ai/e/helion-hackathon/hackathon/gallery/22. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt