Xuan Son Nguyen 0da5d86026 server : allow using LoRA adapters per-request (#10994) ...

* slot.can_batch_with

* lora per request

* test: force disable cache prompt

* move can_batch_with check

* fix condition

* add slow test with llama 8b

* update docs

* move lora change task to queue

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* lora_base

* remove redundant check

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2025-01-02 15:05:18 +01:00

13 KiB

Raw Blame History

View Raw