SIGN IN SIGN UP

FlashInfer: Kernel Library for LLM Serving

0 0 188 Python

Add dynamic tokens-per-page TRTLLM-GEN GQA kernels (#3259)

## Summary
- update TRTLLM-GEN FMHA artifact path and checksum to the newly
published cubins
- select dynamic tokens-per-page kernels for paged GQA decode and
prefill when page size is at least 128
- keep MLA decode kernel selection on dense-mask keys and add
Blackwell-only dynamic page-size coverage
- relax fp16 Blackwell attention-sink context tolerance to match
observed precision noise

## Tests
- pre-commit run on changed files
- python3 -m pytest -q -n 8 --tb=short --maxfail=5 -k
trtllm-gen-mla-filter tests/attention/test_trtllm_gen_mla.py: 7686
passed
- python3 -m pytest -q -n 8 --tb=short
tests/attention/test_attention_sink_blackwell.py: 144 passed


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
  * Updated an artifact path and its checksum manifest.

* **New Features**
* Dynamic page-size selection for generation attention kernels on
supported GPUs.
* Per-token variable sparse-MLA top‑K lengths for finer-grained sparse
attention.
* Expanded MLA-generation kernel support for additional head/dimension
combos.

* **Tests**
  * Added GPU-only tests for dynamic page-size prefill/decode scenarios.
  * Relaxed float16 numerical tolerance in an attention test.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Perkz Zheng <PerkzZheng@users.noreply.github.com>
P
Perkz Zheng committed
4f072c4fdfd6d4e29e7d493c3b191023b7825ba8
Parent: 5ef7afa
Committed by GitHub <noreply@github.com> on 5/11/2026, 7:05:53 AM