Migrate torchao integration off deleted torchao.dtypes (#47797)
torchao removed the torchao/dtypes package (AffineQuantizedTensor and the
Layout classes). This updates the transformers torchao integration to stop
importing from torchao.dtypes so it no longer crashes against current
torchao, mirroring the approach in transformers #45321.
Changes:
- integrations/torchao.py: drop the `_quantization_type` helper and its
`from torchao.dtypes import AffineQuantizedTensor` isinstance branch;
`_linear_extra_repr` now detects quantized weights inline via
`isinstance(self.weight, TorchAOBaseTensor)` (from torchao.utils).
- tests/quantization/torchao_integration/test_torchao.py: replace the
`from torchao.dtypes import AffineQuantizedTensor` import with
`from torchao.utils import TorchAOBaseTensor` and update the isinstance
assertions accordingly.
- docs/source/en/quantization/torchao.md: remove the MarlinSparseLayout
("int4-weight-only-24sparse") examples and migrate the Int4CPULayout
example to `Int4WeightOnlyConfig(int4_packing_format=PLAIN_INT32)`.
Test plan:
- `python -c "import transformers.integrations.torchao"` imports cleanly.
- `RUN_SLOW=1 pytest tests/quantization/torchao_integration/test_torchao.py`:
all 18 TorchAOBaseTensor isinstance / FqnToConfig tests pass. The 3
remaining failures (test_int4wo_quant, test_int4wo_offload,
test_serialization_accelerator_*_Int4WeightOnlyConfig) are pre-existing
generation-output text comparisons pinned to torchao int4 kernel numerics
on the test hardware, unrelated to this change.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> V
Vasiliy Kuznetsov committed
0c98a994d3adfcbbbb9e27b53a9f44dd09453714
Parent: 0104732
Committed by GitHub <noreply@github.com>
on 8/5/2026, 1:52:35 PM