mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2025-10-30 08:42:00 +00:00
08d5986290
* opt performance by reorder for Intel GPU * detect hw type and save opt feature, and print opt feature * correct name * support optimize graph once when compute graph, record the opt status in tensor->extra, make CI passed * add env variable GGML_SYCL_DISABLE_OPT for debug * use syclex::architecture replace the custom hw define, update the guide for GGML_SYCL_DISABLE_OPT * add performance data * mv getrows functions to separeted files * fix global variables --------- Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
llama.cpp/example/sycl
This example program provides the tools for llama.cpp for SYCL on Intel GPU.
Tool
| Tool Name | Function | Status |
|---|---|---|
| llama-ls-sycl-device | List all SYCL devices with ID, compute capability, max work group size, ect. | Support |
llama-ls-sycl-device
List all SYCL devices with ID, compute capability, max work group size, ect.
-
Build the llama.cpp for SYCL for the specified target (using GGML_SYCL_TARGET).
-
Enable oneAPI running environment (if GGML_SYCL_TARGET is set to INTEL -default-)
source /opt/intel/oneapi/setvars.sh
- Execute
./build/bin/llama-ls-sycl-device
Check the ID in startup log, like:
found 2 SYCL devices:
| | | | |Max | |Max |Global | |
| | | | |compute|Max work|sub |mem | |
|ID| Device Type| Name|Version|units |group |group|size | Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]| Intel Arc A770 Graphics| 1.3| 512| 1024| 32| 16225M| 1.3.29138|
| 1| [level_zero:gpu:1]| Intel UHD Graphics 750| 1.3| 32| 512| 32| 62631M| 1.3.29138|