fix(autotuner): differentiate file cache entries by runner specific kernel parameters (#3367)
## ๐ Description
The persistent autotune file cache key was constructed as a 3-tuple
`(custom_op, runner_class, profile)`, intentionally dropping
`hash(runner)` for cross-process stability, but **unintentionally also
dropping `extras`**, which carries runnerspecific parameters like
`use_8x4_sf_layout`, which caused `TrtllmGemmRunner` instances with
`use_8x4_sf_layout=True` and `use_8x4_sf_layout=False` to collide in the
file cache. When vLLM or other frameworks persists and reloads autotune
results, the wrong tactic gets applied, producing:
```
RuntimeError: Check failed: (config.mOptions.mSfLayoutB == mOptions.sfLayoutB)
is false: Invalid sf layout in run
```
**Updates:**
1. **`flashinfer/autotuner.py`**, extend file key from 3-tuple to
4-tuple in `search_cache`, `save_configs`, `load_from_file`.
2. **`flashinfer/gemm/gemm_base.py`**, implement
`get_cache_key_extras()` to return `(self._use_8x4_sf_layout,)`.
3. **`tests/autotuner/test_autotuner_configs.py`**, add
`TestFileCacheKeyCollision` with two tests reproducing and update
existing tests for the new 4-tuple key format.
## ๐ Related Issues
- Fixes flashinfer-ai/flashinfer#3363
- Related to vllm-project/vllm#42537
## ๐ Pull Request Checklist
### โ
Pre-commit Checks
- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.
## ๐งช Tests
- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).
- `test_autotuner_configs.py` โ **37 passed**
- `test_autotuner_core.py` โ **78 passed**
## Reviewer Notes
The bug is latent until autotune results are persisted to disk and
reloaded. It was exposed by vllm-project/vllm#42537 which added
persistent caching for FlashInfer autotuning in vLLM.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed cache key mismatch between memory and disk so runner-specific
parameters (extras/layout) are preserved when saving/loading autotuner
results.
* Resolved collisions where configurations that differed only by
layout-related parameters were treated as the same.
* **Tests**
* Added and updated tests to verify distinct file-cache entries and
loading for entries differing by extras/layout.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/flashinfer-ai/flashinfer/pull/3367?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-lyris01.lyris.clusters.nvidia.com> A
Albert Cheng committed
2f372e2e5f7c1c894c6aed11e4f02fc6dd69ee56
Parent: dac8358
Committed by GitHub <noreply@github.com>
on 5/22/2026, 8:22:46 PM