SIGN IN SIGN UP

ast: stop a rule index path at its last constrained level (#9108)

insertPath walked the whole level list for every rule, inserting an
"absent" node for each level the rule does not constrain. A rule that
constrains nothing below has nothing to test there, so that padding only
gave the rule a private copy of the tail -- and with one distinct ref
per rule, n levels cost n^2 nodes to build and to walk on every lookup:

```rego
p := 1 if input.a == "x"
p := 2 if input.b == "x"
p := 3 if input.c == "x"
```

    trie nodes:   n=10   n=50   n=100   n=250
        before     120   2600   10200   62750
         after      30    150     300     750

Before — 15 nodes
```
  (root)
  input.a ?
  ├─ = "x"
  │  input.b ?          ← rule 1 doesn't mention input.b
  │  └─ absent
  │     input.c ?       ← or input.c
  │     └─ absent
  │        → rule 1
  └─ absent
     input.b ?
     ├─ = "x"
     │  input.c ?       ← rule 2 doesn't mention input.c
     │  └─ absent
     │     → rule 2
     └─ absent
        input.c ?
        └─ = "x"
           → rule 3
```

After — 9 nodes
```
  (root)
  input.a ?
  ├─ = "x"
  │  → rule 1
  └─ absent
     input.b ?
     ├─ = "x"
     │  → rule 2
     └─ absent
        input.c ?
        └─ = "x"
           → rule 3
```

Stop when the path has no constraint left to place. The rule is attached
above the levels it does not constrain, which cannot change what a
lookup selects: traversal always descends the "absent" child, so those
levels never excluded it.

    BenchmarkBuildNakedRefIndex/1000   46.3ms 128.7MB 1007047 allocs
                                    -> 27.4ms   0.8MB    8047 allocs
    BenchmarkBuildNonsensicalIndex/1000  42.5ms 128.9MB
                                      -> 20.3ms   1.0MB
    BenchmarkLookupNakedRefIndex        51.4us -> 33.9us

Policies whose rules constrain the same refs are unaffected -- every
path reaches its last level at the same depth, so there was no padding
to remove.

What this does for a lookup, on 500 rules of `input.subject in
data.groups.gN.members` plus a shared `input.resource.foo == "A"` --
where each group ref is a level of its own:

    500 rules:   33.7ms -> 0.72ms      1000 rules:  136.9ms -> 1.56ms

```rego
p := 1 if { input.subject in data.groups.g0.members; input.resource.foo == "A" }
p := 2 if { input.subject in data.groups.g1.members; input.resource.foo == "A" }
```

  Before — 12 nodes
```
  input.subject ?
  └─ any
     input.resource.foo ?
     └─ = "A"
        data.groups.g0.members ?
        ├─ any
        │  data.groups.g1.members ?   ← re-tested, and re-resolved from the store
        │  └─ absent
        │     → rule 1
        └─ absent
           data.groups.g1.members ?
           └─ any
              → rule 2
```

  After — 10 nodes
```
  input.subject ?
  └─ any
     input.resource.foo ?
     └─ = "A"
        data.groups.g0.members ?
        ├─ any
        │  → rule 1
        └─ absent
           data.groups.g1.members ?
           └─ any
              → rule 2
```

That shape used to be faster with indexing disabled than with it (6.5x
at 500 rules, 8.8x at 1000). It no longer is. 🥳

--------
benchstat
```
                                          │ main      │            PR             │
                                          │ sec/op    │  sec/op     vs base       │
  BuildEqIndex/1-16                          477.3n     478.7n        ~ (p=0.589)
  BuildEqIndex/10-16                         3.175µ     3.174µ        ~ (p=0.818)
  BuildEqIndex/100-16                        31.89µ     32.05µ        ~ (p=0.818)
  BuildEqIndex/1000-16                       402.9µ     401.4µ        ~ (p=0.699)
  LookupEqIndex-16                           163.7n     166.1n        ~ (p=0.093)
  BuildNakedRefIndex/1-16                    384.1n     387.1n        ~ (p=1.000)
  BuildNakedRefIndex/10-16                   7.268µ     4.787µ  -34.14% (p=0.002)
  BuildNakedRefIndex/100-16                  468.4µ     165.1µ  -64.75% (p=0.002)
  BuildNakedRefIndex/1000-16                 48.08m     11.41m  -76.27% (p=0.002)
  LookupNakedRefIndex-16                     53.56µ     33.97µ  -36.58% (p=0.002)
  BuildNonsensicalIndex/1-16                 462.3n     460.7n        ~ (p=0.699)
  BuildNonsensicalIndex/10-16                7.638µ     5.189µ  -32.06% (p=0.002)
  BuildNonsensicalIndex/100-16               477.1µ     143.6µ  -69.90% (p=0.002)
  BuildNonsensicalIndex/1000-16             44.878m     8.886m  -80.20% (p=0.002)
  LookupDistinctRefIndex/10-16              1864.5n     886.0n  -52.48% (p=0.002)
  LookupDistinctRefIndex/100-16            179.913µ     9.580µ  -94.68% (p=0.002)
  LookupDistinctRefIndex/1000-16          18201.4µ      117.5µ  -99.35% (p=0.002)
  RuleIndexRefOrdering/frequency-desc-16     80.67µ     77.72µ        ~ (p=0.260)
  RuleIndexRefOrdering/unordered-16          585.9µ     580.1µ        ~ (p=0.699)
  geomean                                    50.54µ     22.58µ  -55.32%

                                          │ B/op      │  B/op       vs base       │
  BuildNakedRefIndex/1000-16              125636.7Ki   761.2Ki  -99.39% (p=0.002)
  BuildNonsensicalIndex/1000-16           125878.4Ki  1003.4Ki  -99.20% (p=0.002)
  LookupDistinctRefIndex/1000-16             3332.00     36.00  -98.92% (p=0.002)
  BuildEqIndex/* , Lookup*Index               (all unchanged, many all-samples-equal)
  geomean                                    7.021Ki   2.206Ki  -68.57%


                                   │ main       │             PR             │
                                   │ sec/op     │  sec/op     vs base        │
  IndexedRulesetEval/groups=100-16     549.4µ      128.4µ   -76.63% (p=0.002)
  IndexedRulesetEval/groups=500-16   12293.7µ      565.0µ   -95.40% (p=0.002)
  geomean                              2.599m      269.3µ   -89.64%

                                   │ B/op       │  B/op       vs base        │
  IndexedRulesetEval/groups=100-16    172.0Ki     172.4Ki    +0.22%
  IndexedRulesetEval/groups=500-16    855.7Ki     858.4Ki    +0.32%

                                   │ allocs/op  │ allocs/op   vs base        │
  IndexedRulesetEval/groups=100-16     3.195k      3.197k     +0.06%
  IndexedRulesetEval/groups=500-16     15.60k      15.61k     +0.05%
```

---------

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
S
Stephan Renatus committed
7ea3045a582868555ac81355fc92ed3c90493322
Parent: 565a134
Committed by GitHub <noreply@github.com> on 8/31/2026, 2:36:48 PM