Added api for getting/setting the kv_cache (#685)
The api provides access methods for retrieving the current memory buffer for the kv_cache and its token number. It also contains a method for setting the kv_cache from a memory buffer. This makes it possible to load/save history - maybe support --cache-prompt paramater as well? Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
C
Christian Falch committed
e986f94829bae0b9e66b326acbbba179931c84f1
Parent: c0bb1d3
Committed by GitHub <noreply@github.com>
on 4/2/2023, 10:23:04 AM