ast: stop a rule index path at its last constrained level (#9108)
insertPath walked the whole level list for every rule, inserting an
"absent" node for each level the rule does not constrain. A rule that
constrains nothing below has nothing to test there, so that padding only
gave the rule a private copy of the tail -- and with one distinct ref
per rule, n levels cost n^2 nodes to build and to walk on every lookup:
```rego
p := 1 if input.a == "x"
p := 2 if input.b == "x"
p := 3 if input.c == "x"
```
trie nodes: n=10 n=50 n=100 n=250
before 120 2600 10200 62750
after 30 150 300 750
Before — 15 nodes
```
(root)
input.a ?
├─ = "x"
│ input.b ? ← rule 1 doesn't mention input.b
│ └─ absent
│ input.c ? ← or input.c
│ └─ absent
│ → rule 1
└─ absent
input.b ?
├─ = "x"
│ input.c ? ← rule 2 doesn't mention input.c
│ └─ absent
│ → rule 2
└─ absent
input.c ?
└─ = "x"
→ rule 3
```
After — 9 nodes
```
(root)
input.a ?
├─ = "x"
│ → rule 1
└─ absent
input.b ?
├─ = "x"
│ → rule 2
└─ absent
input.c ?
└─ = "x"
→ rule 3
```
Stop when the path has no constraint left to place. The rule is attached
above the levels it does not constrain, which cannot change what a
lookup selects: traversal always descends the "absent" child, so those
levels never excluded it.
BenchmarkBuildNakedRefIndex/1000 46.3ms 128.7MB 1007047 allocs
-> 27.4ms 0.8MB 8047 allocs
BenchmarkBuildNonsensicalIndex/1000 42.5ms 128.9MB
-> 20.3ms 1.0MB
BenchmarkLookupNakedRefIndex 51.4us -> 33.9us
Policies whose rules constrain the same refs are unaffected -- every
path reaches its last level at the same depth, so there was no padding
to remove.
What this does for a lookup, on 500 rules of `input.subject in
data.groups.gN.members` plus a shared `input.resource.foo == "A"` --
where each group ref is a level of its own:
500 rules: 33.7ms -> 0.72ms 1000 rules: 136.9ms -> 1.56ms
```rego
p := 1 if { input.subject in data.groups.g0.members; input.resource.foo == "A" }
p := 2 if { input.subject in data.groups.g1.members; input.resource.foo == "A" }
```
Before — 12 nodes
```
input.subject ?
└─ any
input.resource.foo ?
└─ = "A"
data.groups.g0.members ?
├─ any
│ data.groups.g1.members ? ← re-tested, and re-resolved from the store
│ └─ absent
│ → rule 1
└─ absent
data.groups.g1.members ?
└─ any
→ rule 2
```
After — 10 nodes
```
input.subject ?
└─ any
input.resource.foo ?
└─ = "A"
data.groups.g0.members ?
├─ any
│ → rule 1
└─ absent
data.groups.g1.members ?
└─ any
→ rule 2
```
That shape used to be faster with indexing disabled than with it (6.5x
at 500 rules, 8.8x at 1000). It no longer is. 🥳
--------
benchstat
```
│ main │ PR │
│ sec/op │ sec/op vs base │
BuildEqIndex/1-16 477.3n 478.7n ~ (p=0.589)
BuildEqIndex/10-16 3.175µ 3.174µ ~ (p=0.818)
BuildEqIndex/100-16 31.89µ 32.05µ ~ (p=0.818)
BuildEqIndex/1000-16 402.9µ 401.4µ ~ (p=0.699)
LookupEqIndex-16 163.7n 166.1n ~ (p=0.093)
BuildNakedRefIndex/1-16 384.1n 387.1n ~ (p=1.000)
BuildNakedRefIndex/10-16 7.268µ 4.787µ -34.14% (p=0.002)
BuildNakedRefIndex/100-16 468.4µ 165.1µ -64.75% (p=0.002)
BuildNakedRefIndex/1000-16 48.08m 11.41m -76.27% (p=0.002)
LookupNakedRefIndex-16 53.56µ 33.97µ -36.58% (p=0.002)
BuildNonsensicalIndex/1-16 462.3n 460.7n ~ (p=0.699)
BuildNonsensicalIndex/10-16 7.638µ 5.189µ -32.06% (p=0.002)
BuildNonsensicalIndex/100-16 477.1µ 143.6µ -69.90% (p=0.002)
BuildNonsensicalIndex/1000-16 44.878m 8.886m -80.20% (p=0.002)
LookupDistinctRefIndex/10-16 1864.5n 886.0n -52.48% (p=0.002)
LookupDistinctRefIndex/100-16 179.913µ 9.580µ -94.68% (p=0.002)
LookupDistinctRefIndex/1000-16 18201.4µ 117.5µ -99.35% (p=0.002)
RuleIndexRefOrdering/frequency-desc-16 80.67µ 77.72µ ~ (p=0.260)
RuleIndexRefOrdering/unordered-16 585.9µ 580.1µ ~ (p=0.699)
geomean 50.54µ 22.58µ -55.32%
│ B/op │ B/op vs base │
BuildNakedRefIndex/1000-16 125636.7Ki 761.2Ki -99.39% (p=0.002)
BuildNonsensicalIndex/1000-16 125878.4Ki 1003.4Ki -99.20% (p=0.002)
LookupDistinctRefIndex/1000-16 3332.00 36.00 -98.92% (p=0.002)
BuildEqIndex/* , Lookup*Index (all unchanged, many all-samples-equal)
geomean 7.021Ki 2.206Ki -68.57%
│ main │ PR │
│ sec/op │ sec/op vs base │
IndexedRulesetEval/groups=100-16 549.4µ 128.4µ -76.63% (p=0.002)
IndexedRulesetEval/groups=500-16 12293.7µ 565.0µ -95.40% (p=0.002)
geomean 2.599m 269.3µ -89.64%
│ B/op │ B/op vs base │
IndexedRulesetEval/groups=100-16 172.0Ki 172.4Ki +0.22%
IndexedRulesetEval/groups=500-16 855.7Ki 858.4Ki +0.32%
│ allocs/op │ allocs/op vs base │
IndexedRulesetEval/groups=100-16 3.195k 3.197k +0.06%
IndexedRulesetEval/groups=500-16 15.60k 15.61k +0.05%
```
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com> S
Stephan Renatus committed
7ea3045a582868555ac81355fc92ed3c90493322
Parent: 565a134
Committed by GitHub <noreply@github.com>
on 8/31/2026, 2:36:48 PM