Handle invalid CUDA JITerator cache entries (#186346)
JITerator trusted any readable persistent kernel cache file. If a cache entry was truncated or otherwise invalid, CUDA module loading failed with a driver-level invalid argument error and PyTorch surfaced that failure directly instead of rebuilding the kernel. This matched the reported torch.prod failure mode where clearing ~/.cache/torch/kernels made the error disappear. When loading a cached CUDA kernel fails, remove that cache entry, warn once, and fall back to the normal NVRTC compilation path. Also close and validate the per-process temporary cache file before renaming it into the final cache path so other processes cannot observe an incomplete final file from this writer. The regression test creates a prod JITerator cache entry, truncates it, and verifies a second process recompiles successfully using the same cache path. I chose recovery at cache-load time rather than disabling JITerator caching or special-casing prod, because the cache validity problem is shared by all JITerator users. Fixes #132756 Generated by my agent Test Plan: - ninja -C build - python test/test_jiterator.py TestPythonJiteratorCUDA.test_corrupt_kernel_cache_recompiles_cuda - lintrunner -a Pull Request resolved: https://github.com/pytorch/pytorch/pull/186346 Approved by: https://github.com/frgossen
J
Jason Ansel committed
a82aae9d4a7827849ce50f31c4c7ee8f278d05f5
Parent: 047d461
Committed by PyTorch MergeBot <pytorchmergebot@users.noreply.github.com>
on 8/25/2026, 11:41:20 PM