DPI Research

Not sure where the f8_quant submission is supposed to go, so here it is: https://github.com/dpiresearch/Helion_20260314/blob/main/helion/fp8_quant_py/submission.py For causal_conf1d Tuned the Helion execution config so the kernel uses the GPU more effectively: num_warps: 1 → 4 More threads per block (4×32 = 128), so more parallelism and better occupancy. num_stages: 1 → 2 More pipeline stages so load/store and compute can overlap better (better memory latency hiding). For the gated_deltanet_* kernels Here’s a concise summary of what the gated_deltanet\* submission kernels improve over the reference implementations. 1. gated_deltanet_chunk_fwd_h Reference (reference.py): PyTorch eager, sequential over chunks. For each chunk it: Keeps full chunk tensors k_c, w_c, u_c, g_c in memory Does v_new_c[:, c] = u_c[:, c] - w_c[:, c] @ h and h = h * exp(g_last) + k_c[:, c].T @ v_gated Loops over c in Python; lots of intermediates and global memory traffic Improvements in submission (submission.py): Single Helion kernel with explicit tiling: tiles over (B*H, V) with block [1, 8] and over time T with chunk size C = 64. Recurrent state in registers: keeps state [K, V] per (b, h) and updates it chunk-by-chunk with hl.dot for the matmuls (e.g. w @ state, k_adj.T @ diff), avoiding full chunk-sized matmuls in global memory. Specialized sizes: K and V are hl.specialize(...) so the compiler can optimize for fixed dimensions. Stable config: One helion.Config (e.g. num_warps=4, num_stages=2) for all shapes to stay within the leaderboard timeout while still being GPU-friendly. So the main improvement is moving from a chunk-by-chunk Python loop with big PyTorch matmuls to one tiled Helion kernel that keeps the recurrence in small state and uses hl.dot for the core math. 2. gated_deltanet_chunk_fwd_o Reference: Reshapes to chunks and does: o_inter = (q_c @ h) * exp(g_c) (inter-chunk) Full C×C qk = q_c @ k_c.T * exp(g_diff) with causal mask, then o = (o_inter + qk @ v_c) * scale Large temporary tensors for the full chunk–chunk attention. Improvements in submission: Tiled over (BH, T) with block [1, C] so each program works on one chunk of time for one (b, h). Same math, better mapping: Intra-chunk: qk = hl.dot(q_tile, k_tile.T), causal mask and g_diff, then sim = where(causal, qk * exp(g_diff), 0) and local_out = hl.dot(sim, v_tile). Inter-chunk: global_out = hl.dot(q_s, h[c_idx]) with q_s = q_tile * exp(g_vals). Output: (global_out + local_out) * scale. Helion dot primitives instead of large PyTorch matmuls, so the work is expressed as smaller, cache-friendly ops. Config: e.g. num_warps=8, num_stages=4 for more parallelism on this matmul-heavy kernel. So the improvement is preserving the exact chunk-fwd-o formula while executing it in a tiled, dot-based way that fits the GPU and avoids huge intermediates. 3. gated_deltanet_recompute_w_u Reference: Reshapes to (B, NT, C, H, K/V) and does two batched matmuls: u_c = A_c @ (v_c * beta_c) w_c = A_c @ (k_c * (beta_c * exp(g_c))) then permute/reshape back to (B, T, H, K/V). Improvements in submission: No big batched matmuls: the matmul is expressed as two explicit passes over the inner dimension ci in 0..C-1 and ci in C-1..0, each accumulating: w_acc += a_col[:, None] * (k_ci * coeff_ci * decay_ci)[None, :] u_acc += a_col[:, None] * (v_ci * coeff_ci)[None, :] Averaging: w_out = (w_acc1 + w_acc2) * 0.5, u_out = (u_acc1 + u_acc2) * 0.5. That gives a symmetric (forward+backward) order of summation, which can improve numerical behavior (e.g. cancellation) compared to a single pass. Tiling: over (B*H, T) with block [1, C], so each tile handles one chunk and the inner loop is over C with small accumulators (hl.zeros([rt, K]), hl.zeros([rt, V])), which can stay in registers or fast memory. Config: e.g. num_warps=4, num_stages=2 for stability under the 12-minute leaderboard limit. So the improvement is replacing one-shot batched matmuls with a tiled, two-pass accumulation that is both GPU-friendly and numerically more stable. Cross-cutting improvements Static shapes and configs: All three use static_shapes=True and SHAPE_CONFIGS keyed by (B, T, H, K, V) so the right kernel/config is chosen per shape without autotuning on the bot. Single config for timeout: Comments note that B200-tuned or autotuned configs hit the 12-minute leaderboard timeout; using one “safe” config per kernel improves reliability. IEEE dot precision: All use dot_precision="ieee" for correctness. Optional ACF: Comments point to advanced_controls_file for further tuning (e.g. booster pack ACFs) once a baseline is correct and stable. In short: the gated_deltanet submissions improve over the references by turning chunk-wise PyTorch loops and large matmuls into single, tiled Helion kernels that use small state, explicit dots, and (for recompute_w_u) a two-pass summed form for better numerics and GPU utilization.