CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-27 08:21:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8d8862829cd770fc9c0c7a726ed162de824ff5ea
llama.cpp/CODEOWNERS
Line: 4: incorrect codeowner user: ngxson Line: 4: no users/groups matched Line: 5: incorrect codeowner user: slaren Line: 5: incorrect codeowner user: CISC Line: 5: no users/groups matched Line: 6: incorrect codeowner user: CISC Line: 6: no users/groups matched Line: 7: incorrect codeowner user: slaren Line: 7: no users/groups matched Line: 8: incorrect codeowner user: slaren Line: 8: no users/groups matched Line: 9: incorrect codeowner user: ggerganov Line: 9: no users/groups matched Line: 10: incorrect codeowner user: ggerganov Line: 10: no users/groups matched Line: 11: incorrect codeowner user: ggerganov Line: 11: no users/groups matched Line: 12: incorrect codeowner user: ggerganov Line: 12: incorrect codeowner user: ericcurtin Line: 12: no users/groups matched Line: 13: incorrect codeowner user: ggerganov Line: 13: no users/groups matched Line: 14: incorrect codeowner user: ggerganov Line: 14: no users/groups matched Line: 15: incorrect codeowner user: ggerganov Line: 15: no users/groups matched Line: 16: incorrect codeowner user: ggerganov Line: 16: no users/groups matched Line: 17: incorrect codeowner user: angt Line: 17: no users/groups matched Line: 18: incorrect codeowner user: ggerganov Line: 18: no users/groups matched Line: 19: incorrect codeowner user: ggerganov Line: 19: no users/groups matched Line: 20: incorrect codeowner user: ggerganov Line: 20: no users/groups matched Line: 21: incorrect codeowner user: ggerganov Line: 21: no users/groups matched Line: 22: incorrect codeowner user: CISC Line: 22: no users/groups matched Line: 23: incorrect codeowner user: ggerganov Line: 23: no users/groups matched Line: 24: incorrect codeowner user: ggerganov Line: 24: no users/groups matched Line: 25: incorrect codeowner user: ggerganov Line: 25: no users/groups matched Line: 26: incorrect codeowner user: ggerganov Line: 26: no users/groups matched Line: 27: incorrect codeowner user: am17an Line: 27: no users/groups matched Line: 28: incorrect codeowner user: ggerganov Line: 28: no users/groups matched Line: 29: incorrect codeowner user: ggerganov Line: 29: no users/groups matched Line: 30: incorrect codeowner user: ggerganov Line: 30: no users/groups matched Line: 31: incorrect codeowner user: ggerganov Line: 31: no users/groups matched Line: 32: incorrect codeowner user: ggerganov Line: 32: no users/groups matched Line: 33: incorrect codeowner user: ggerganov Line: 33: no users/groups matched Line: 34: incorrect codeowner user: ggerganov Line: 34: no users/groups matched Line: 35: incorrect codeowner user: ggerganov Line: 35: no users/groups matched Line: 36: incorrect codeowner user: ggerganov Line: 36: no users/groups matched Line: 37: incorrect codeowner user: JohannesGaessler Line: 37: no users/groups matched Line: 38: incorrect codeowner user: danbev Line: 38: no users/groups matched Line: 39: incorrect codeowner user: ggerganov Line: 39: no users/groups matched Line: 40: incorrect codeowner user: ggerganov Line: 40: no users/groups matched Line: 41: incorrect codeowner user: ggerganov Line: 41: no users/groups matched Line: 42: incorrect codeowner user: ggerganov Line: 42: no users/groups matched Line: 43: incorrect codeowner user: slaren Line: 43: no users/groups matched Line: 44: incorrect codeowner user: slaren Line: 44: no users/groups matched Line: 45: incorrect codeowner user: ggerganov Line: 45: no users/groups matched Line: 46: incorrect codeowner user: ggerganov Line: 46: no users/groups matched Line: 47: incorrect codeowner user: ggerganov Line: 47: no users/groups matched Line: 48: incorrect codeowner user: ggerganov Line: 48: incorrect codeowner user: slaren Line: 48: no users/groups matched Line: 49: incorrect codeowner user: slaren Line: 49: no users/groups matched Line: 50: incorrect codeowner user: slaren Line: 50: no users/groups matched Line: 51: incorrect codeowner user: slaren Line: 51: no users/groups matched Line: 52: incorrect codeowner user: ggerganov Line: 52: incorrect codeowner user: slaren Line: 52: no users/groups matched Line: 53: incorrect codeowner user: ggerganov Line: 53: incorrect codeowner user: slaren Line: 53: no users/groups matched Line: 54: incorrect codeowner user: alex-spacemit Line: 54: no users/groups matched Line: 55: incorrect codeowner user: slaren Line: 55: no users/groups matched Line: 56: incorrect codeowner user: JohannesGaessler Line: 56: no users/groups matched Line: 57: incorrect codeowner user: slaren Line: 57: no users/groups matched Line: 58: incorrect codeowner user: JohannesGaessler Line: 58: incorrect codeowner user: am17an Line: 58: no users/groups matched Line: 59: incorrect codeowner user: JohannesGaessler Line: 59: no users/groups matched Line: 60: incorrect codeowner user: JohannesGaessler Line: 60: no users/groups matched Line: 61: incorrect codeowner user: JohannesGaessler Line: 61: no users/groups matched Line: 62: incorrect codeowner user: IMbackK Line: 62: no users/groups matched Line: 63: incorrect codeowner user: IMbackK Line: 63: no users/groups matched Line: 64: incorrect codeowner user: IMbackK Line: 64: no users/groups matched Line: 65: incorrect codeowner user: ggerganov Line: 65: incorrect codeowner user: slaren Line: 65: no users/groups matched Line: 66: incorrect codeowner user: ggerganov Line: 66: no users/groups matched Line: 67: incorrect codeowner user: lhez Line: 67: incorrect codeowner user: max-krasnyansky Line: 67: no users/groups matched Line: 68: incorrect codeowner user: max-krasnyansky Line: 68: no users/groups matched Line: 69: incorrect codeowner user: JohannesGaessler Line: 69: no users/groups matched Line: 70: incorrect codeowner user: ggerganov Line: 70: no users/groups matched Line: 71: incorrect codeowner user: rgerganov Line: 71: no users/groups matched Line: 72: incorrect codeowner user: ggerganov Line: 72: incorrect codeowner user: slaren Line: 72: no users/groups matched Line: 73: incorrect codeowner user: 0cc4m Line: 73: no users/groups matched Line: 74: incorrect codeowner user: reeselevine Line: 74: no users/groups matched Line: 75: incorrect codeowner user: taronaeo Line: 75: incorrect codeowner user: Andreas-Krebbel Line: 75: incorrect codeowner user: AlekseiNikiforovIBM Line: 75: no users/groups matched Line: 76: incorrect codeowner user: ggerganov Line: 76: incorrect codeowner user: slaren Line: 76: no users/groups matched Line: 77: incorrect codeowner user: ggerganov Line: 77: incorrect codeowner user: slaren Line: 77: no users/groups matched Line: 78: incorrect codeowner user: JohannesGaessler Line: 78: incorrect codeowner user: Green-Sky Line: 78: no users/groups matched Line: 79: incorrect codeowner user: CISC Line: 79: no users/groups matched Line: 80: incorrect codeowner user: ggerganov Line: 80: no users/groups matched Line: 81: incorrect codeowner user: ggerganov Line: 81: no users/groups matched Line: 82: incorrect codeowner user: ggerganov Line: 82: no users/groups matched Line: 83: incorrect codeowner user: ggerganov Line: 83: no users/groups matched Line: 84: incorrect codeowner user: ggerganov Line: 84: no users/groups matched Line: 85: incorrect codeowner user: CISC Line: 85: no users/groups matched Line: 86: incorrect codeowner user: CISC Line: 86: no users/groups matched Line: 87: incorrect codeowner user: ngxson Line: 87: no users/groups matched Line: 88: incorrect codeowner user: CISC Line: 88: no users/groups matched Line: 89: incorrect codeowner user: slaren Line: 89: no users/groups matched Line: 90: incorrect codeowner user: CISC Line: 90: no users/groups matched Line: 91: incorrect codeowner user: CISC Line: 91: no users/groups matched Line: 92: incorrect codeowner user: ggerganov Line: 92: no users/groups matched Line: 93: incorrect codeowner user: slaren Line: 93: no users/groups matched Line: 94: incorrect codeowner user: slaren Line: 94: no users/groups matched Line: 95: incorrect codeowner user: ggerganov Line: 95: no users/groups matched Line: 96: incorrect codeowner user: slaren Line: 96: no users/groups matched Line: 97: incorrect codeowner user: ggerganov Line: 97: no users/groups matched Line: 98: incorrect codeowner user: ngxson Line: 98: no users/groups matched Line: 99: incorrect codeowner user: ggerganov Line: 99: no users/groups matched Line: 100: incorrect codeowner user: ggerganov Line: 100: no users/groups matched Line: 101: incorrect codeowner user: rgerganov Line: 101: no users/groups matched Line: 102: incorrect codeowner user: ericcurtin Line: 102: no users/groups matched Line: 103: incorrect codeowner user: ngxson Line: 103: incorrect codeowner user: ggerganov Line: 103: incorrect codeowner user: ericcurtin Line: 103: no users/groups matched Line: 104: incorrect codeowner user: allozaur Line: 104: no users/groups matched Line: 105: incorrect codeowner user: ggerganov Line: 105: no users/groups matched Line: 106: incorrect codeowner user: ggerganov Line: 106: no users/groups matched Line: 107: incorrect codeowner user: ggerganov Line: 107: no users/groups matched Line: 108: incorrect codeowner user: slaren Line: 108: no users/groups matched Line: 109: incorrect codeowner user: slaren Line: 109: no users/groups matched Line: 110: incorrect codeowner user: ggerganov Line: 110: no users/groups matched Line: 111: incorrect codeowner user: ggerganov Line: 111: no users/groups matched Line: 112: incorrect codeowner user: ggerganov Line: 112: no users/groups matched Line: 113: incorrect codeowner user: ggerganov Line: 113: no users/groups matched Line: 114: incorrect codeowner user: ggerganov Line: 114: no users/groups matched Line: 115: incorrect codeowner user: ggerganov Line: 115: no users/groups matched Line: 116: incorrect codeowner user: danbev Line: 116: no users/groups matched Line: 117: incorrect codeowner user: CISC Line: 117: no users/groups matched
Max Krasnyansky 63d2fc46e1 Add experimental ggml-hexagon backend for the Hexagon NPU (#16547) 
* model: add support for extra bufs for all devices

* hexagon: add experimental ggml-hexagon backend for the Hexagon NPU

This commit introduces a new experimental backend `ggml-hexagon` with support for the Hexagon NPU.

Highlights:
- Supports Hexagon versions: v73, v75, v79, and v81
- Targets Android devices based on Snapdragon SoCs: Gen3, 8-Elite, and 8-Elite Gen5
- Supports Q4_0, Q8_0, MXFP4, and FP32 data types
- Implements core LLM ops: MUL_MAT/MUL_MAT_ID, ADD/SUB/MUL/ADD_ID, RMS_NORM, ROPE, GLU/SWIGLU, SOFTMAX

**Note:** This backend is experimental and may exhibit instability or limited performance across supported devices.
It is intended for early testing and feedback from llama.cpp/ggml developer and user community.

Co-Authored-By: Rajdeep Ganguly <rganguly@qti.qualcomm.com>
Co-Authored-By: Todor Boinovski <todorb@qti.qualcomm.com>

* hexagon: fix format checker errors

* hexagon: update readme and cmake presets

* ci: add android-ndk-build jobs that build plain ARM64 and Snapdragon versions

* hexagon: add simple graph optimizer for stacking MUL_MAT ops with the same input

* hexagon: move ADB helper scripts into scripts/snapdragon/adb

* hexagon: replace all f/printfs with GGML_LOG_...

* readme: add hexagon to the list supported backends

* hexagon: stack malmuts with quantized inputs only

* hexagon: add TODO for fixing issues in hexagon_graph_optimize

* hexagon: update to hex-sdk 6.4.0 and add scripts for running on QDC

* scripts: fix lint errors

* scripts: update qdc pytest script to make linter happy

* hexagon: add reduce sum in fp32

* hexagon: reduce number of vector stores in matmul output

* hexagon: remove the need for vdelta in reduce-multiply-x8

* hexagon: consistent use of reduce_sum_fp32 for row_sums

* hexagon: some more matmul optimizations and comments

Optimize cases where tensor dims are not multiple of 1024 (e.g in Qwen models).
We've handled those cases already but at a higher overhead.

* hexagon: update cmake presets

* hexagon: add OPMASK support for run-bench.sh wrapper

* hexagon: update to use GGML_BACKEND_API

* hexagon: remove unused logic for setting tensor flags for the views

* hexagon: add asserts to set/get_tensor to make sure we handle complete tensors

Same asserts as the CPU backend.

* hexagon: use cpy_tensor slow path for non-host buffers

* hexagon: error checks in the buffer allocator

* cmake: move include(extProj) under ggml-hexagon

* hexagon: don't forget to delete the backend on free

* hexagon: set/get_tensor size assert apply only to quantized tensors

* hexagon: reintroduce HEX_VERBOSE wrapper for GGML_LOG_DEBUG for now

GGML_LOG_DEBUG is always enabled for test-backend-ops and the output gets in the way.
Ideally we need a bit more finer log levels.

* docs: typos in hexagon developer docs (libggm-...)

* hexagon: overhaul error handling in the session/device allocation

this should handle all failure paths in the session allocation.

* hexagon: update cmake presets to enable fp16 vectors

* hexagon: remove unused time_usec function

* hexagon: don't forget to release buffer contexts

* hexagon: fixed indents in hvx-utils (missed clang-format auto-format failure)

* hexagon: remove custom can_repeat function and use ggml_can_repeat

---------

Co-authored-by: Rajdeep Ganguly <rganguly@qti.qualcomm.com>
Co-authored-by: Todor Boinovski <todorb@qti.qualcomm.com>
2025-10-22 13:47:09 -07:00

118 lines
5.9 KiB
Plaintext
Raw Blame History

# collaborators can optionally add themselves here to indicate their availability for reviewing related PRs
# multiplie collaborators per item can be specified
/.devops/*.Dockerfile @ngxson
/.github/actions/ @slaren @CISC
/.github/workflows/ @CISC
/.github/workflows/release.yml @slaren
/.github/workflows/winget.yml @slaren
/ci/ @ggerganov
/cmake/ @ggerganov
/common/CMakeLists.txt @ggerganov
/common/arg.* @ggerganov @ericcurtin
/common/base64.hpp.* @ggerganov
/common/build-info.* @ggerganov
/common/common.* @ggerganov
/common/console.* @ggerganov
/common/http.* @angt
/common/llguidance.* @ggerganov
/common/log.* @ggerganov
/common/sampling.* @ggerganov
/common/speculative.* @ggerganov
/convert_*.py @CISC
/examples/batched.swift/ @ggerganov
/examples/batched/ @ggerganov
/examples/convert-llama2c-to-ggml/ @ggerganov
/examples/deprecation-warning/ @ggerganov
/examples/diffusion/ @am17an
/examples/embedding/ @ggerganov
/examples/eval-callback/ @ggerganov
/examples/export-docs/ @ggerganov
/examples/gen-docs/ @ggerganov
/examples/gguf/ @ggerganov
/examples/llama.android/ @ggerganov
/examples/llama.swiftui/ @ggerganov
/examples/llama.vim @ggerganov
/examples/lookahead/ @ggerganov
/examples/lookup/ @JohannesGaessler
/examples/model-conversion/ @danbev
/examples/parallel/ @ggerganov
/examples/passkey/ @ggerganov
/examples/retrieval/ @ggerganov
/examples/save-load-state/ @ggerganov
/examples/simple-chat/ @slaren
/examples/simple/ @slaren
/examples/speculative-simple/ @ggerganov
/examples/speculative/ @ggerganov
/ggml/cmake/ @ggerganov
/ggml/include/ @ggerganov @slaren
/ggml/src/ggml-alloc.c @slaren
/ggml/src/ggml-backend* @slaren
/ggml/src/ggml-blas/ @slaren
/ggml/src/ggml-common.h @ggerganov @slaren
/ggml/src/ggml-cpu/ @ggerganov @slaren
/ggml/src/ggml-cpu/spacemit/ @alex-spacemit
/ggml/src/ggml-cuda/common.cuh @slaren
/ggml/src/ggml-cuda/fattn* @JohannesGaessler
/ggml/src/ggml-cuda/ggml-cuda.cu @slaren
/ggml/src/ggml-cuda/mmf.* @JohannesGaessler @am17an
/ggml/src/ggml-cuda/mmq.* @JohannesGaessler
/ggml/src/ggml-cuda/mmvf.* @JohannesGaessler
/ggml/src/ggml-cuda/mmvq.* @JohannesGaessler
/ggml/src/ggml-cuda/fattn-wmma* @IMbackK
/ggml/src/ggml-hip/ @IMbackK
/ggml/src/ggml-cuda/vendors/hip.h @IMbackK
/ggml/src/ggml-impl.h @ggerganov @slaren
/ggml/src/ggml-metal/ @ggerganov
/ggml/src/ggml-opencl/ @lhez @max-krasnyansky
/ggml/src/ggml-hexagon/ @max-krasnyansky
/ggml/src/ggml-opt.cpp @JohannesGaessler
/ggml/src/ggml-quants.* @ggerganov
/ggml/src/ggml-rpc/ @rgerganov
/ggml/src/ggml-threading.* @ggerganov @slaren
/ggml/src/ggml-vulkan/ @0cc4m
/ggml/src/ggml-webgpu/ @reeselevine
/ggml/src/ggml-zdnn/ @taronaeo @Andreas-Krebbel @AlekseiNikiforovIBM
/ggml/src/ggml.c @ggerganov @slaren
/ggml/src/ggml.cpp @ggerganov @slaren
/ggml/src/gguf.cpp @JohannesGaessler @Green-Sky
/gguf-py/ @CISC
/media/ @ggerganov
/scripts/gen* @ggerganov
/scripts/get* @ggerganov
/scripts/sync* @ggerganov
/src/ @ggerganov
/src/llama-adapter.* @CISC
/src/llama-arch.* @CISC
/src/llama-chat.* @ngxson
/src/llama-graph.* @CISC
/src/llama-model-loader.* @slaren
/src/llama-model.* @CISC
/src/llama-vocab.* @CISC
/tests/ @ggerganov
/tests/test-backend-ops.cpp @slaren
/tests/test-thread-safety.cpp @slaren
/tools/batched-bench/ @ggerganov
/tools/llama-bench/ @slaren
/tools/main/ @ggerganov
/tools/mtmd/ @ngxson
/tools/perplexity/ @ggerganov
/tools/quantize/ @ggerganov
/tools/rpc/ @rgerganov
/tools/run/ @ericcurtin
/tools/server/* @ngxson @ggerganov @ericcurtin # no subdir
/tools/server/webui/ @allozaur
/tools/tokenize/ @ggerganov
/tools/tts/ @ggerganov
/vendor/ @ggerganov
/.clang-format @slaren
/.clang-tidy @slaren
/AUTHORS @ggerganov
/CMakeLists.txt @ggerganov
/CONTRIBUTING.md @ggerganov
/LICENSE @ggerganov
/README.md @ggerganov
/SECURITY.md @ggerganov
/build-xcframework.sh @danbev
requirements*.txt @CISC
Reference in New Issue View Git Blame Copy Permalink