[Router] Add prefix_min_match_length threshold to PrefixAwareRouter (#959)
Add a configurable --prefix-min-match-length option for the prefixaware routing logic. When the longest prefix match is shorter than this threshold, the request falls back to QPS-based routing instead of using the matched endpoint, mitigating load hotspots caused by long shared prefixes (e.g. common system prompts). Defaults to 0, which preserves the original behavior. Partially addresses #957. Signed-off-by: Max Li <hitliqiwei@gmail.com> Co-authored-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com>
Y
ywc668 committed
e0c1a2ea72acd42566e702b26b783c406abf8e11
Parent: 9b78da7
Committed by GitHub <noreply@github.com>
on 6/22/2026, 8:49:44 PM