SIGN IN SIGN UP

Added api for getting/setting the kv_cache (#685)

The api provides access methods for retrieving the current memory buffer for the kv_cache and its token number.
It also contains a method for setting the kv_cache from a memory buffer.

This makes it possible to load/save history - maybe support --cache-prompt paramater as well?

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
C
Christian Falch committed
e986f94829bae0b9e66b326acbbba179931c84f1
Parent: c0bb1d3
Committed by GitHub <noreply@github.com> on 4/2/2023, 10:23:04 AM