server : fix crash when system prompt is bigger than batch size (#5714)
The system prompt is now decoded in batches. * server : fix off-by-one n_past when start of prompt matches whole cache The tokens right after the matching part would otherwise skip a pos value.
C
compilade committed
f7625019c51ca437a5840576d92362cfa710e4a2
Parent: abbabc5
Committed by GitHub <noreply@github.com>
on 2/25/2024, 6:43:50 PM