Christian Falch e986f94829 Added api for getting/setting the kv_cache (#685) ...

The api provides access methods for retrieving the current memory buffer for the kv_cache and its token number.
It also contains a method for setting the kv_cache from a memory buffer.

This makes it possible to load/save history - maybe support --cache-prompt paramater as well?

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>

2023-04-02 12:23:04 +02:00

5.8 KiB

Raw Blame History

View Raw