llama.cpp/examples/server/server.cpp at e39e727e9a3daec7082b9c59540a429ad85914af

mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-31 08:51:55 +00:00

Files

Benson Wong 5d01670266 server : include speculative decoding stats when timings_per_token is enabled (#12603 )

* Include speculative decoding stats when timings_per_token is true

New fields added to the `timings` object:

  - draft_n           : number of draft tokens generated
  - draft_accepted_n  : number of draft tokens accepted
  - draft_accept_ratio: ratio of accepted/generated

* Remove redundant draft_accept_ratio var

* add draft acceptance rate to server console output

2025-03-28 10:05:44 +02:00

180 KiB

Raw Blame History

View Raw

180 KiB Raw Blame History

180 KiB

Raw Blame History