Building a High-Performance and Portable vLLM Linear Backend with Helion – PyTorch

A new Helion-based linear backend for vLLM unifies Standard GEMM, Split-K, and Swap-AB kernel variants into a single autotuned implementation targeting NVIDIA Hopper GPUs. Using per-shape autotuning and a hybrid dispatch strategy (Helion under CUDA Graph replay for small num_tokens, falling back to CUTLASS/DeepGEMM for larger shapes), the backend achieves geometric mean kernel speedups of 1.11x-1.18x over default CUTLASS, FlashInfer, and DeepGEMM kernels across FP8_Dynamic, W8A8_INT8, and Block_FP8 quantization formats. End-to-end serving benchmarks across Qwen3 model sizes show consistent throughput gains exceeding 10% for some workloads. The piece also discusses a performance-usability-maintainability tradeoff triangle, LLM-guided autotuning search using Claude Opus 4.8, and a proposed upstream adoption model where Helion kernels ship with default configs while workload-specific tuning is delegated to end users.

Source: https://pytorch.org/blog/building-a-high-performance-and-portable-vllm-linear-backend-with-helion. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.