mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2025-11-21 12:16:57 +00:00
1920345c3b
* Add files via upload
* fix unit test
* fix crashes for --reasoning-format=none
* Patch buggy official MiniMax-M2 chat template
* add upstream minja fix: https://github.com/ochafik/minja/pull/7
* Fix <think> token not generated
* add test copied from https://github.com/ggml-org/llama.cpp/pull/16946
* cleanup
* Hopes to fix the compilation error on CI
* Delete chat template patching since it’s fixed by upstream Minja
* Remove undeeded Minimax-M2 template patch
https://github.com/ochafik/minja/pull/7#issuecomment-3480356100
* Add proper handling of optional parameters with test
merged tests from: 23d4bb75c4
* Fix making all tool parameters optional
* Move xml tool parser to separate file
* cleanup & add tests for GLM4.5
* add streaming tests & enhancement & cleanups
Add streaming test for both GLM 4.5 and minimax-m2.
Cleanup for preserved_tokens.
Cleanup for grammar rule name.
Enhance the parser's stability.
* cleanup & add support for Kimi-K2 Qwen3-Coder Apriel-1.5 Xiaomi-MiMo
* apply suggestions from reviewers
* fix a misuse for data.grammar_lazy
* fix grammar when tool have no argument
* Fix `no triggers set for lazy grammar!` for GLM4.5/4.6. Insert additional stops for Kimi-K2
* update chat.cpp
* fix grammar for GLM 4.5/4.6
* Try fix Jinja template for GLM
* Try fix GLM-4.6.jinja
* Update common/chat-parser-xml-toolcall.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update tests/test-chat.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* improve chat template for GLM, rename Kimi-K2 template to Kimi-K2-Thinking
* Improve Kimi-K2 chat template
* Fix unit test
* Fix "Invalid tool call arguments passed" in a rare case.
In a rare case, the model may emit a raw string that begins with a valid JSON string. This commit adds unit tests to cover that scenario and fixes the regression introduced during the Kimi-K2 adaptation.
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
107 lines
3.3 KiB
Django/Jinja
107 lines
3.3 KiB
Django/Jinja
[gMASK]<sop>
|
|
{%- if tools -%}
|
|
<|system|>
|
|
# Tools
|
|
|
|
You may call one or more functions to assist with the user query.
|
|
|
|
You are provided with function signatures within <tools></tools> XML tags:
|
|
<tools>
|
|
{% for tool in tools %}
|
|
{{ tool | tojson(ensure_ascii=False) }}
|
|
{% endfor %}
|
|
</tools>
|
|
|
|
For each function call, output the function name and arguments within the following XML format:
|
|
<tool_call>{function-name}
|
|
<arg_key>{arg-key-1}</arg_key>
|
|
<arg_value>{arg-value-1}</arg_value>
|
|
<arg_key>{arg-key-2}</arg_key>
|
|
<arg_value>{arg-value-2}</arg_value>
|
|
...
|
|
</tool_call>{%- endif -%}
|
|
{%- macro visible_text(content) -%}
|
|
{%- if content is string -%}
|
|
{{- content }}
|
|
{%- elif content is iterable and content is not mapping -%}
|
|
{%- for item in content -%}
|
|
{%- if item is mapping and item.type == 'text' -%}
|
|
{{- item.text }}
|
|
{%- elif item is string -%}
|
|
{{- item }}
|
|
{%- endif -%}
|
|
{%- endfor -%}
|
|
{%- else -%}
|
|
{{- content }}
|
|
{%- endif -%}
|
|
{%- endmacro -%}
|
|
{%- set ns = namespace(last_user_index=-1) %}
|
|
{%- for m in messages %}
|
|
{%- if m.role == 'user' %}
|
|
{% set ns.last_user_index = loop.index0 -%}
|
|
{%- endif %}
|
|
{%- endfor %}
|
|
{% for m in messages %}
|
|
{%- if m.role == 'user' -%}<|user|>
|
|
{{ visible_text(m.content) }}
|
|
{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
|
|
{%- elif m.role == 'assistant' -%}
|
|
<|assistant|>
|
|
{%- set reasoning_content = '' %}
|
|
{%- set content = visible_text(m.content) %}
|
|
{%- if m.reasoning_content is string %}
|
|
{%- set reasoning_content = m.reasoning_content %}
|
|
{%- else %}
|
|
{%- if '</think>' in content %}
|
|
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
|
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
|
{%- endif %}
|
|
{%- endif %}
|
|
{%- if loop.index0 > ns.last_user_index and reasoning_content -%}
|
|
{{ '\n<think>' + reasoning_content.strip() + '</think>'}}
|
|
{%- else -%}
|
|
{{ '\n<think></think>' }}
|
|
{%- endif -%}
|
|
{%- if content.strip() -%}
|
|
{{ '\n' + content.strip() }}
|
|
{%- endif -%}
|
|
{% if m.tool_calls %}
|
|
{% for tc in m.tool_calls %}
|
|
{%- if tc.function %}
|
|
{%- set tc = tc.function %}
|
|
{%- endif %}
|
|
{{ '\n<tool_call>' + tc.name }}
|
|
{% set _args = tc.arguments or {} %}
|
|
{% if _args is not mapping %}
|
|
{{ raise_exception("Invalid tool call arguments passed: " + _args | string) }}
|
|
{% endif %}
|
|
{% for k, v in _args.items() %}
|
|
<arg_key>{{ k }}</arg_key>
|
|
<arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>
|
|
{% endfor %}
|
|
</tool_call>{% endfor %}
|
|
{% endif %}
|
|
{%- elif m.role == 'tool' -%}
|
|
{%- if m.content is string -%}
|
|
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
|
{{- '<|observation|>' }}
|
|
{%- endif %}
|
|
{{- '\n<tool_response>\n' }}
|
|
{{- m.content }}
|
|
{{- '\n</tool_response>' }}
|
|
{%- else -%}
|
|
<|observation|>{% for tr in m.content %}
|
|
|
|
<tool_response>
|
|
{{ tr.output if tr.output is defined else tr }}
|
|
</tool_response>{% endfor -%}
|
|
{% endif -%}
|
|
{%- elif m.role == 'system' -%}
|
|
<|system|>
|
|
{{ visible_text(m.content) }}
|
|
{%- endif -%}
|
|
{%- endfor -%}
|
|
{%- if add_generation_prompt -%}
|
|
<|assistant|>{{- '\n<think></think>' if (enable_thinking is defined and not enable_thinking) else '' -}}
|
|
{%- endif -%}
|