[TableTransformer, PI0] Fix stale expected values and OOM in integration tests (#48198)

* [WIP] Fix integration test failures: TableTransformer, T5 compile, PI0

- TableTransformer: stale expected logit/box values for A10G (sm_86)
- T5: CUDAGraphs tensor overwrite in test_compile_static_cache, likely
  introduced by #47731 (stop setting static cache as model attribute)
- PI0: OOM in test_train_pi0_base_libero (float32 training on 22 GiB GPU)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [TableTransformer] Fix stale expected values for A10G (sm_86)

Add ("cuda", 8) entries to expected_logits_data and expected_boxes_data
in test_table_detection, captured on A10G (compute capability 8.6).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Fix OOM in test_train_pi0_base_libero by using bfloat16

The test loaded pi0_base (PaLiGemma-3B-based) in float32 and ran full
Trainer training, which OOMs on a 22 GiB A10G GPU. Switch to bfloat16
(torch_dtype=torch.bfloat16 + TrainingArguments(bf16=True)) to halve
the memory footprint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [TableTransformer] Use (None, None) fallback key instead of ("cuda", None)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Fix OOM in test_train_pi0_base_libero (bfloat16 + adafactor)

The test loaded pi0_base (PaLiGemma-3B backbone) in float32 with default
Adam optimizer, which OOMs on a 22 GiB A10G GPU: Adam stores exp_avg +
exp_avg_sq (~12 GB) on top of the model weights (~6 GB) and gradients
(~6 GB), totalling ~24 GB regardless of batch size.

Fix:
- torch_dtype=torch.bfloat16 (halves model footprint)
- optim="adafactor" (factored 2nd moment, no 1st moment; ~0.3 GB vs 12 GB)
- per_device_train_batch_size=2 (reduces activation memory)
- max_steps=6 (one extra step for a clearer trend)
- Relax the loss assertion to allow 1 non-monotone step: with a small
  batch size and bf16 precision, occasional gradient noise can cause a
  single step to tick up even as the overall trend is downward.

Verified on A10G runner (22 GiB).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Add comment explaining why adafactor is used

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Fix dtype mismatch on multi-GPU machines (DataParallel + bfloat16)

On multi-GPU runners Trainer wraps the model in nn.DataParallel, which
scatters float32 dataset inputs to each GPU replica. The model weights
are bfloat16, causing a dtype mismatch in the first linear layer.

Setting device_map={"": torch_device} causes from_pretrained to record
model.hf_device_map, which Trainer checks to skip DataParallel wrapping.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Extend device_map comment to clarify single-device intent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Fix DataParallel dtype mismatch on multi-GPU runners

Replace device_map approach (which only skips DataParallel when the
model spans multiple GPUs) with explicitly setting trainer.args._n_gpu=1
before training. This reliably prevents Trainer from wrapping the model
in nn.DataParallel regardless of how many GPUs are available.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* [PI0] Use @require_torch_non_multi_accelerator to skip on multi-GPU runners

On multi-GPU machines Trainer wraps the model in nn.DataParallel, which
scatters float32 dataset inputs to replicas while the model weights are
bfloat16, causing a dtype mismatch. This test is not intended to cover
multi-GPU training, so skip it on multi-GPU setups via the decorator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>