fix(gdn): allow importing gdn_decode without a CUDA device (#3293)
<!-- .github/pull_request_template.md -->
## ๐ Description
- Defer `torch.cuda.get_device_properties()` and
`torch.cuda.get_device_capability()` in `gdn_decode_bf16_state.py `from
module scope to lazy `@functools.cache` helpers that derive the device
from the input tensor (q.device), matching the convention used by the
CuTe DSL MLA and MoE backends.
- Catch RuntimeError in addition to ImportError in all gdn_decode.py
backend import blocks, matching the existing pattern in
`gdn_kernels/__init__.py.`
Before this PR:
```
CUDA_VISIBLE_DEVICES= python3 -c "import flashinfer.gdn_decode; print('Success: import worked without CUDA devices')"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/workspace/flashinfer/flashinfer/gdn_decode.py", line 58, in <module>
from .gdn_kernels.gdn_decode_bf16_state import (
File "/workspace/flashinfer/flashinfer/gdn_kernels/gdn_decode_bf16_state.py", line 1389, in <module>
NUM_SMS = torch.cuda.get_device_properties(0).multi_processor_count
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py312/lib/python3.12/site-packages/torch/cuda/__init__.py", line 614, in get_device_properties
_lazy_init() # will define _get_device_properties
^^^^^^^^^^^^
File "/opt/conda/envs/py312/lib/python3.12/site-packages/torch/cuda/__init__.py", line 410, in _lazy_init
torch._C._cuda_init()
RuntimeError: No CUDA GPUs are available
```
After this PR:
```
CUDA_VISIBLE_DEVICES= python3 -c "import flashinfer.gdn_decode; print('Success: import worked without CUDA devices')"
Success: import worked without CUDA devices
```
## ๐ Related Issues
https://github.com/flashinfer-ai/flashinfer/issues/3262
## ๐ Pull Request Checklist
Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.
### โ
Pre-commit Checks
- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.
> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).
## ๐งช Tests
- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).
## Reviewer Notes
<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Decode backends now treat runtime import failures like import errors
and gracefully fall back to safe defaults.
* **Performance / Reliability**
* GPU kernels adapt at runtime to the actual device, selecting
device-aware configurations and maintaining separate compiled kernels
per device for more reliable and optimized execution.
[](https://app.coderabbit.ai/change-stack/flashinfer-ai/flashinfer/pull/3293)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Cursor <cursoragent@cursor.com> K
Ka-Hyun Nam committed
885d0cf4803f99a04574aa7777a74d50dd713e20
Parent: 89bfb68
Committed by GitHub <noreply@github.com>
on 5/14/2026, 9:53:56 PM