mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2025-11-03 09:22:01 +00:00
f5cd27b71d
* add common_json w/ support for truncated json healing * add common_chat_msg_diff * partial common_chat_parse * refactor parser w/ optionals * server: wire chat diffs in stream mode * fix trigger of thinking models (must happen after thoughts are closed) * fix functionary v3.2 raw python! * rename: common_chat_syntax (now contains format) * rm common_regex.at_start * don't return empty <think></think> * accommodate yet another deepseek r1 distill fantasy syntax (`<｜tool▁calls｜>`) * fix QwQ 32B tool call parsing after thoughts (hermes2) * better logs for grammar triggers * consume spaces after parse_json_tool_calls * fix required tool calls w/ thinking models that have pre-opened thinking tags * fix thinking model's initial trigger + test qwq's template * run most test_tool_call tests in stream + non-stream modes * make functionary v3.2 parsing more strict (differentiate first match from others) * send final diff from server, to close off raw python arguments * support partial content streaming in Generic mode * tool-call: allow content prelude before hermes2 tool calls (for Qwen2.5) * Update function-calling.md * Update tool_bench.py * chat-parser: remove input from exception (llm output may contain PII) --------- Co-authored-by: ochafik <ochafik@google.com> Co-authored-by: Olivier Chafik <ochafik@users.noreply.github.com>
38 lines
1.7 KiB
C++
38 lines
1.7 KiB
C++
#pragma once
|
|
#include <json.hpp>
|
|
|
|
// Healing marker (empty if the JSON was fully parsed / wasn't healed).
|
|
struct common_healing_marker {
|
|
// Raw marker.
|
|
std::string marker;
|
|
|
|
// Cutting the `common_json.json.dump()` string at the (only) occurrence of this marker should yield the original partial JSON string (modulo spaces / if it had the same dump format).
|
|
std::string json_dump_marker;
|
|
};
|
|
|
|
// Represents a parsed JSON object, with its optional healing marker (a JSON dump fragment that can be used to find the position of healing in the JSON dump string)
|
|
struct common_json {
|
|
nlohmann::ordered_json json;
|
|
|
|
common_healing_marker healing_marker;
|
|
};
|
|
|
|
// Parse the JSON string, healing (closing) any partial JSON if `healing_marker` is not empty.
|
|
//
|
|
// Healing completes partial JSON strings by adding a (possibly modified) healing marker, then whatever is needed to close the JSON.
|
|
// This allows to parse the resulting healed JSON string, yet be able to cut it again if needed at the healing marker.
|
|
// (this is used when parsing JSON outputs from the models, then crafting partial JSONs for the partial tool calls in OAI format).
|
|
//
|
|
// For instance, parsing `{` with a healing marker `foo` will produce a healed JSON `{"foo":1}`, w/ json_dump_marker = `"foo"` (which can be used to break the JSON again).
|
|
bool common_json_parse(
|
|
const std::string & input,
|
|
const std::string & healing_marker,
|
|
common_json & out);
|
|
|
|
// Parse the JSON string (see overload above), but advancing an iterator to the end of the input when the (potentially partial) parsing succeeds.
|
|
bool common_json_parse(
|
|
std::string::const_iterator & it,
|
|
const std::string::const_iterator & end,
|
|
const std::string & healing_marker,
|
|
common_json & out);
|