SIGN IN SIGN UP

[inductor] AOTI: route integer-scalar / integral-tensor ops off the c-shim to the proxy executor (#194262) (#194262)

Summary:

Under AOTInductor all-fallback (lite mode), an op with an integer Scalar arg on an integral
tensor -- e.g. `aten.add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1)` with an int64
`self` -- fails at RUNTIME through the c-shim:

    RuntimeError: For integral input tensors, argument alpha must not be a floating point number.

Root cause: the torchgen'd AOTI c-shim types every `Scalar` parameter as `double` (to carry
both int and float through one ABI). So the integer `alpha=1` is coerced to `1.0` at the shim
boundary, and ATen's `alpha_check` rejects a floating-point alpha for integral tensors. Normal
lowering never hits this because `add.Tensor` is an inlined pointwise op (no ATen dispatch, no
`c10::Scalar` at runtime); only all-fallback emits the opaque c-shim ATen call.

Fix: generalize the existing "complex scalar can't fit the double shim ABI -> use the proxy
executor" special-case in `FallbackKernel.codegen` into a shared predicate
`_cshim_scalar_abi_forces_proxy`. A scalar bound to a Number/Scalar param must leave the c-shim
when it is:
  - complex -- does not fit in a double at all (the prior special-case, now subsumed);
  - bool    -- see below; or
  - int, and an integral tensor is present -- the `alpha=1` -> `1.0` case above.
Such ops are routed to the proxy executor, which serializes each scalar with its real type, so
ATen reconstructs the right kind of `c10::Scalar` and the checks pass.

Booleans are handled as their own case, and deliberately without the integral-tensor gate. A
bool is a distinct kind of `c10::Scalar` (`Scalar::isBoolean()`), and the double ABI erases
that tag -- `True` arrives as `1.0`, a Double Scalar that is neither boolean nor integral. Two
separate things break, neither covered by the integer rule:
  - `alpha_check` (ATen/native/BinaryOps.h:18-22) rejects it for a Bool result, because a
    Double is not integral -- whereas eager accepts a Boolean alpha, which counts as integral
    under `isIntegral(includeBool=true)`;
  - wherever the scalar feeds dtype inference the result changes: `aten.full.default` infers
    bool from `full(size, True)` but float from `full(size, 1.0)`, so the runtime output stops
    matching the layout inductor inferred from the real bool.
That second case has no tensor argument at all, so `has_integral_tensor` has nothing to gate
on -- hence no gate. It is latent rather than live today, because `full.default`'s c-shim is
version-gated (`"since": TORCH_VERSION_2_10_0`) and currently absent, so the op already reaches
the proxy executor via the no-c-shim rule; the bool rule is what keeps it correct once that
shim lands. Bool scalars are rare, so routing all of them costs effectively nothing, and the
round trip is already exact: `serialize.py` emits `as_bool` (checked ahead of `as_int`, since
`bool` subclasses `int`) and the `NumberType` branch of the proxy executor rebuilds a boolean
IValue. Note `type(value) is int` rather than `isinstance` in the integer rule, so bool does
not fall through to it.

The predicate is wired into BOTH:
- `FallbackKernel.codegen` (sets `use_runtime_dispatch`); and
- `_materialize_scalar_tensor_args` (the Option B materialization gate) -- so a routed op's
  scalar-in-Tensor-slot args (e.g. the `other` in `add.Tensor(x, 1)`) are materialized to real
  constant buffers too; otherwise `fill_args` re-hits "got <class 'int'>" for the scalar in a
  Tensor slot. Both sites must use the same predicate.

Note: `caffe2/...` and `xplat/caffe2/...` are separate synced copies; both are edited.

Test Plan:
In `caffe2/test/inductor/test_aot_inductor.py` (fbcode + xplat), all via `check_model` with
`options={"fallback_by_default": True}` -- compile, package, load, run, compare against eager:

- `test_proxy_executor_integral_add_scalar` -- `add.Tensor(int64_tensor, 1)`; previously raised
  the integral-alpha error at runtime.
- `test_proxy_executor_integral_pow_scalar` -- `pow.Tensor_Scalar(int64_tensor, 2)`, the same
  rule via an explicit Scalar slot rather than a schema default.
- `test_proxy_executor_bool_scalar_alpha` -- NEW. `add.Tensor(bool_tensor, bool_tensor,
  alpha=True)`. Verified to be a real regression test: with the bool rule disabled it fails
  with the predicted
      RuntimeError: For integral input tensors, argument alpha must not be a floating point number.
  and passes with it.
- `test_proxy_executor_bool_scalar_infers_dtype` -- NEW. `full.default(x.shape, True,
  device=x.device) & x`, covering the no-tensor-argument dtype-inference case. Labelled in the
  test as a forward-looking guard, NOT a regression test: it passes with and without the bool
  rule today because full.default has no c-shim in this build, and starts exercising the rule
  once that shim lands.

```
buck2 test @//mode/opt fbcode//caffe2/test/inductor:test_aot_inductor \
  -- --regex "proxy_executor|scalar_tensor|integral"
Tests finished: Pass 40. Fail 0. Skip 13.
```

Because the bool rule widens which ops leave the c-shim, the cpp-wrapper suite was re-run as a
regression check:

```
buck2 test @//mode/opt-amd-gpu fbcode//caffe2/test/inductor:test_inductor \
  -- --env TORCHINDUCTOR_CPP_WRAPPER=1
```
No new failures versus the parent commit (the residual failures on this devserver are
pre-existing: hipBLASLt HIPBLAS_STATUS_INVALID_VALUE, FlashAttention dtype, and an undefined
`torch_new_stable_ivalue` symbol).

`lintrunner` clean on both changed files.

Also validated end-to-end on the recsys `merge` net (AMD MI350, gfx950) `--mode fallback`:
with this fix (on top of the walls 1-7 fixes) the model FULLY lowers, packages, loads, and
RUNS -- `max_abs_diff_vs_eager=3.9e-3` (matching the normal-lowering baseline), 52 outputs.

Authored with an AI assistant.

Reviewed By: desertfire

Differential Revision: D114083095

Pull Request resolved: https://github.com/pytorch/pytorch/pull/194262
Approved by: https://github.com/sevenEng, https://github.com/desertfire
Z
Zhuoran Zhao committed
7e8b24611a4eb1e6cb3971ab343da47cf9c2a66f
Parent: caca41c
Committed by PyTorch MergeBot <pytorchmergebot@users.noreply.github.com> on 8/23/2026, 10:34:19 PM