CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-28 08:31:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

mtmd : support InternVL 2.5 and 3 (#13422)

Browse Source 
* convert : internvl support

* InternVL3-1B working

* fix regression

* rm mobilevlm from test

* fix conversion

* add test for internvl

* add to list of pre-quant

* restore boi/eoi check

* add clarify comment for norm eps
This commit is contained in:
Xuan-Son Nguyen
2025-05-10 16:26:42 +02:00
committed by GitHub
parent d8919424f1
commit 053367d149
9 changed files with 243 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
tools/mtmd/mtmd.cpp
View File

@@ -252,6 +252,13 @@ int32_t mtmd_tokenize(mtmd_context * ctx,
}
else if (proj_type == PROJECTOR_TYPE_INTERNVL) {
// <img> ... (image embeddings) ... </img>
marker_modified = "<img>" + ctx->image_marker + "</img>";
string_replace_all(prompt_modified, ctx->image_marker, marker_modified);
}
// llava-1.5, llava-1.6, Yi-VL, Yi-34B, granite: don't need to add prefix and suffix
// for glm-edge, BOI and EOI token's embeddings are not present in the text model