Skip to Main Content

usagi

Built at PyTorch Helion Hackathon · Mar 14, 2026 · San Francisco, CA

implemented all 4 scored Helion kernels for Gated DeltaNet and causal conv1d, optimized for B200 GPUs with per-shape autotuned configs. causal_conv1d: Zero-copy causal depthwise conv: eliminated external padding via in-kernel index clamping/masking, avoiding redundant memory traffic from torch.cat. ~2.9x speedup over naive padded approach. TMA + 6-stage pipelining, hl.specialize(W) for compile-time unrolling. gated_deltanet_chunk_fwd_h: Inter-chunk state recurrence with sequential scan, maintaining [K,V] hidden state in registers. hl.dot with accumulator for state updates, multi-stage pipelining, tuned loop orders, variable static_ranges handling. gated_deltanet_recompute_w_u: WY-transform kernel computing w and u as two fused dot products sharing a single A matrix load. Mixed tensor_descriptor/pointer indexing, hl.specialize for K, V, and chunk dims. gated_deltanet_chunk_fwd_o: Output kernel combining inter-chunk (q@h) and intra-chunk (causal attention). Early inter-chunk computation for better pipelining, causal masking via hl.arange, persistent_interleaved pid with num_sm_multiplier=8 for full SM utilization.

Team