SIGN IN SIGN UP

FlashInfer: Kernel Library for LLM Serving

0 0 188 Python

feat(cute_dsl/moe): re-enable use_cold_l2_cache in CuteDslMoEWrapper TuningConfig (#3384)

## ๐Ÿ“Œ Description

Sets `use_cold_l2_cache=True` on the autotuner `TuningConfig` in
`flashinfer/fused_moe/cute_dsl/tuner.py`, matching TRT-LLM's
`CuteDslFusedMoENvfp4Runner.tuning_config`. With cold-L2 ON, the
autotuner flushes L2 between profile iterations and measures
conservative timings, so the picked tactic is robustly fast under
cold-cache conditions; without it, back-to-back iterations of the same
tactic benefit from L2-hit reuse and bias the pick toward tactics that
look fast during profiling but aren't faster in production.

## ๐Ÿ” Related Issues

https://github.com/flashinfer-ai/flashinfer/pull/3286
https://github.com/flashinfer-ai/flashinfer/pull/3340

## ๐Ÿš€ Pull Request Checklist

Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.

### โœ… Pre-commit Checks

- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.

> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).

## ๐Ÿงช Tests

- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).

## Reviewer Notes

<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved tuner's cold L2 cache measurement behavior for more accurate
performance profiling.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/flashinfer-ai/flashinfer/pull/3384?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
L
Lee Nau committed
41e5aa29a0e218bde2e2045316ed88e3f4dc8ca2
Parent: bc4dc97
Committed by GitHub <noreply@github.com> on 5/21/2026, 10:48:49 PM