mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2025-10-31 08:51:55 +00:00
5656d10599
* MPI support, first cut * fix warnings, update README * fixes * wrap includes * PR comments * Update CMakeLists.txt * Add GH workflow, fix test * Add info to README * mpi : trying to move more MPI stuff into ggml-mpi (WIP) (#2099) * mpi : add names for layer inputs + prep ggml_mpi_graph_compute() * mpi : move all MPI logic into ggml-mpi Not tested yet * mpi : various fixes - communication now works but results are wrong * mpi : fix output tensor after MPI compute (still not working) * mpi : fix inference * mpi : minor * Add OpenMPI to GH action * [mpi] continue-on-error: true * mpi : fix after master merge * [mpi] Link MPI C++ libraries to fix OpenMPI * tests : fix new llama_backend API * [mpi] use MPI_INT32_T * mpi : factor out recv / send in functions and reuse * mpi : extend API to allow usage with outer backends (e.g. Metal) --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Examples for input embedding directly
Requirement
build libembdinput.so
run the following comman in main dir (../../).
make
LLaVA example (llava.py)
- Obtian LLaVA model (following https://github.com/haotian-liu/LLaVA/ , use https://huggingface.co/liuhaotian/LLaVA-13b-delta-v1-1/).
- Convert it to ggml format.
llava_projection.pthis pytorch_model-00003-of-00003.bin.
import torch
bin_path = "../LLaVA-13b-delta-v1-1/pytorch_model-00003-of-00003.bin"
pth_path = "./examples/embd_input/llava_projection.pth"
dic = torch.load(bin_path)
used_key = ["model.mm_projector.weight","model.mm_projector.bias"]
torch.save({k: dic[k] for k in used_key}, pth_path)
- Check the path of LLaVA model and
llava_projection.pthinllava.py.
PandaGPT example (panda_gpt.py)
- Obtian PandaGPT lora model from https://github.com/yxuansu/PandaGPT. Rename the file to
adapter_model.bin. Use convert-lora-to-ggml.py to convert it to ggml format. Theadapter_config.jsonis
{
"peft_type": "LORA",
"fan_in_fan_out": false,
"bias": null,
"modules_to_save": null,
"r": 32,
"lora_alpha": 32,
"lora_dropout": 0.1,
"target_modules": ["q_proj", "k_proj", "v_proj", "o_proj"]
}
- Papare the
vicunav0 model. - Obtain the ImageBind model.
- Clone the PandaGPT source.
git clone https://github.com/yxuansu/PandaGPT
- Install the requirement of PandaGPT.
- Check the path of PandaGPT source, ImageBind model, lora model and vicuna model in panda_gpt.py.
MiniGPT-4 example (minigpt4.py)
- Obtain MiniGPT-4 model from https://github.com/Vision-CAIR/MiniGPT-4/ and put it in
embd-input. - Clone the MiniGPT-4 source.
git clone https://github.com/Vision-CAIR/MiniGPT-4/
- Install the requirement of PandaGPT.
- Papare the
vicunav0 model. - Check the path of MiniGPT-4 source, MiniGPT-4 model and vicuna model in
minigpt4.py.