SIGN IN SIGN UP

FlashInfer: Kernel Library for LLM Serving

0 0 188 Python

Enable smaller tile N for SM100 Cute-DSL NVFP4 GEMM (#3403)

<!-- .github/pull_request_template.md -->

## 📌 Description

Use tile N = 8, 16, 32

<img width="1540" height="1260" alt="image"
src="https://github.com/user-attachments/assets/5de82572-e8d6-4659-8097-1514ce09746a"
/>

Note: The speedup of M = 1 is actually not due to narrow N (as it could
not use swapAB), but actually because the autotuner is consistently
picking incorrect tactic. Just hardcode it in these cases

<!-- What does this PR do? Briefly describe the changes and why they’re
needed. -->

## 🔍 Related Issues
https://github.com/flashinfer-ai/flashinfer/issues/3402

<!-- Link any related issues here -->

Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.

### ✅ Pre-commit Checks

- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.

> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).

## 🧪 Tests

- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).

on SM103:
```
pytest tests/gemm/test_mm_fp4.py
========================================================================= test session starts ==========================================================================
platform linux -- Python 3.12.3, pytest-9.0.3, pluggy-1.6.0
rootdir: /sgl-workspace/flashinfer
configfile: pytest.ini
plugins: anyio-4.13.0, typeguard-4.5.1
collected 12072 items  
...
tests/gemm/test_mm_fp4.py: 310 warnings
  /sgl-workspace/flashinfer/flashinfer/gemm/kernels/dense_blockscaled_gemm_sm100.py:1365: DeprecationWarning: The 'alignment' parameter of CooperativeGroup's constructor is deprecated and will be removed in a subsequent release, please remove it from your code.
    c_producer_group = pipeline.CooperativeGroup(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================== 3168 passed, 8904 skipped, 2230 warnings in 576.26s (0:09:36) =====================================================
```

## Reviewer Notes

<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Improvements**
* Broader SM100 block‑scaled GEMM tiling options and refined selection
to better handle small‑N and medium‑N workloads while skipping invalid
small‑N choices.
* Added stricter runtime validation to reject unsupported or inefficient
kernel configurations and enforce additional constraints for certain
tiling cases.

* **Tests**
* Expanded FP4 GEMM test coverage with additional m dimension values to
exercise the new tiling and validation paths.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/flashinfer-ai/flashinfer/pull/3403?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
B
Brayden Zhong committed
ed2d443242e810c5840a3cd2a1b945b10c8862f0
Parent: 47cbd2c
Committed by GitHub <noreply@github.com> on 5/27/2026, 11:52:49 PM