Make cute dsl mxfp8/nvfp4 quantizer bitwise exact (#3387)
<!-- .github/pull_request_template.md --> ## ๐ Description @humansand We want to make sure FlashInfer quantization backends are bitwise identical with TE style implementation for RL use cases. Currently cute dsl mxfp8 backend is not bitwise identical on subnormal edge cases, and cute dsl nvfp4 backend does not honor `TRTLLM_DISABLE_FP4_QUANT_FAST_MATH` env var and has fast math always enabled. - MXFP8 - Added MXFP8 bitwise-exact reference checks against reference implementation. - Fixed MXFP8 CuTe scale conversion edge cases and host scale-output initialization. - NVFP4 - Expanded NVFP4 TE-reference exactness coverage to include the CuTe DSL backend. - Skipped unsupported CuTe NVFP4 modes in the TE-reference test: per-token activation and 4over6. - Made CuTe NVFP4 honor `TRTLLM_DISABLE_FP4_QUANT_FAST_MATH=1` for bitwise-exact math. <!-- What does this PR do? Briefly describe the changes and why theyโre needed. --> ## ๐ Related Issues - Bitwise per-tensor NVFP4 quantizer for CUDA backend inplemented in: - https://github.com/flashinfer-ai/flashinfer/pull/3264 - Bitwise per-token NVFP4 quantizer for CUDA backend implemented in: - https://github.com/flashinfer-ai/flashinfer/pull/3027 - Env var `TRTLLM_DISABLE_FP4_QUANT_FAST_MATH ` introduced by: - https://github.com/flashinfer-ai/flashinfer/pull/3237 <!-- Link any related issues here --> ## ๐ Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### โ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## ๐งช Tests - [x] Tests have been added or updated as needed. - [x] All tests are passing (`unittest`, etc.). ## Reviewer Notes <!-- Optional: anything you'd like reviewers to focus on, concerns, etc. --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Optional precise NVFP4 quantization mode (env-controlled) that uses round-to-nearest math, with new RN reciprocal/scale helpers and per-kernel switch for fast-math behavior. * **Bug Fixes** * Fixed an uninitialized buffer in the FP8 quantization CPU path. * **Tests** * Tightened validation to require exact parity, improved mismatch diagnostics, added an extreme-scale MXFP8 test and reference utilities, and backend parametrization. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/flashinfer-ai/flashinfer/pull/3387?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Z
Ziang Li committed
7ac0b5f2989db8ea70cc2ad3caf54f5e705b672e
Parent: c802a05
Committed by GitHub <noreply@github.com>
on 5/27/2026, 4:13:14 PM