CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-31 08:51:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,244 Commits 494 Branches 4,666 Tags
fe86282efe69b9479eb021d7390d49999afdbbdc
Commit Graph

6244 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi Gerganov
b467702b87 readme : fix typos 2023-03-29 19:38:31 +03:00
Georgi Gerganov
516d88e75c readme : add GPT4All instructions (close #588) 2023-03-29 19:37:20 +03:00
Georgi Gerganov
53635c081c py : add GPT4All conversion script 
For now: copy-paste
Too much time for me to deduplicate the python code
 2023-03-29 19:29:52 +03:00
Maël Kerbiriou
41318d708e llama : use the same threshold for OpenBLAS and ggml thread limiting (#577) 2023-03-29 19:10:07 +03:00
Tobias Lütke
a6956b25a1 add example of re-act pattern (#583) 
* add example of re-act pattern

* spelling...

* fixed whitespace in reverse prompt issue
 2023-03-29 10:10:24 -05:00
anzz1
83df5639eb Fix GCC warning about binary literal (#595) 
0b10101010 -> 0xAA /* 0b10101010 */
master-83df563 		2023-03-29 13:20:07 +00:00
anzz1
a5c42c4b13 Fix typo in llama.h (#593) master-a5c42c4 2023-03-29 13:19:29 +00:00
anzz1
5a5f8b1501 Enable Fused-Multiply-Add (FMA) and F16C/CVT16 vector extensions on MSVC (#375) 
* Enable Fused-Multiply-Add (FMA) instructions on MSVC

__FMA__ macro does not exist in MSVC

* Enable F16C/CVT16 vector extensions on MSVC

__F16C__ macro does not exist in MSVC, but is implied with AVX2/AVX512

* MSVC cvt intrinsics

* Add __SSE3__ macro for MSVC too because why not

even though it's not currently used for anything when AVX is defined
master-5a5f8b1 		2023-03-28 22:44:29 +03:00
anzz1
f1217055ea CI: fix subdirectory path globbing (#546) 
- Changes in subdirectories will now be detecter properly
- (Windows-MSVC) AVX512 tests temporarily disabled
master-f121705 		2023-03-28 22:43:25 +03:00
anzz1
7f4c5c6651 llama : fix linkage with mingw (#551) 
* Revert 7e53955 (#542)

Still needs to be fixed properly

* Fix linking on mingw32
master-7f4c5c6 		2023-03-28 21:23:09 +03:00
slaren
2a98bc18ea ggml : add AVX2 implementation of quantize_row_q4_1 (#515) 
* Add AVX2 implementation of quantize_row_q4_1

* Actually use AVX2

* Make quantize_row_q4_1 static

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
master-2a98bc1 		2023-03-28 21:06:03 +03:00
thement
d0aaff571c py : add temporary script to convert old ggml files to newer version (#539) 
Co-authored-by: Jakub Horak <jakub.horak@ibawizard.net>
master-d0aaff5 		2023-03-28 20:55:42 +03:00
Tai Duc Nguyen
d0330fd783 py : add capabiliy to convert from ggml back to torch or hf format for further consumption/training/finetuning (#403) 2023-03-28 20:51:29 +03:00
Stephan Walter
99c5b27654 ggml : refactor quantized processing functions (#509) 
* Refactor quantized processing functions

* ggml : minor

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
master-99c5b27 		2023-03-28 20:13:01 +03:00
DooWoong Lee (David)
692ce3164e py : removed unused model variable and verified that the code functions correctly with vocab_only setting. Also confirmed that the code works as expected after running with reduced memory usage due to deletion of no-longer-needed variable. (#547) 2023-03-28 20:02:34 +03:00
Georgi Gerganov
96f9c0506f ci : make ctest verbose, hopefully we see what is wrong with the sanitizer master-96f9c05 2023-03-28 20:01:09 +03:00
Georgi Gerganov
d502bc7c9d tests : free llama context at the end of the test master-d502bc7 2023-03-28 19:51:55 +03:00
Stephan Walter
436e561931 all : be more strict about converting float to double (#458) 
* Be more strict about converting float to double

* Test equivalence of round, SILU implementations

Test module is commented out in CMakeLists.txt because the tests may
take a long time, depending on how much the compiler optimizes.

* Fix softmax in perplexity.cpp

* all : prefer float over double where appropriate

* perplexity : add <cmath>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
master-436e561 		2023-03-28 19:48:20 +03:00
Jed Fox
20e1e84884 deploy : add a Package.swift for SwiftPM support (#393) 
* Add a Package.swift for SwiftPM support

* Swap from exclusions to allowlist
master-20e1e84 		2023-03-28 19:39:01 +03:00
Stephan Walter
c1f885067c ggml : introduce structs for the q4 data blocks (#356) 
* Introduce structs for the q4 data blocks

* ggml : rename quant struct variables + fix ARM_NEON

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
master-c1f8850 		2023-03-28 18:56:03 +03:00
Georgi Gerganov
e0670260fb gitignore : add "embedding" 2023-03-28 18:34:35 +03:00
dotpy314
28ba975aea Check the existence of f16_model_path_base in quantize.py (#574) 
Co-authored-by: Jincheng Miao <jincheng.miao@gmail.com>
 2023-03-28 18:06:28 +03:00
slaren
a6bdc47cba Fix usage of F16C intrinsics in AVX code (#563) 
* Fix usage of F16C intrinsics in AVX code when F16C is not defined
master-a6bdc47 		2023-03-28 17:26:55 +03:00
anzz1
7b8dbcb78b main.cpp fixes, refactoring (#571) 
- main: entering empty line passes back control without new input in interactive/instruct modes
- instruct mode: keep prompt fix
- instruct mode: duplicate instruct prompt fix
- refactor: move common console code from main->common
master-7b8dbcb 		2023-03-28 17:09:55 +03:00
RJ Adriaansen
4b8efff0e3 Add embedding example to Makefile (#540) master-4b8efff 2023-03-28 09:11:09 +03:00
Marco Matthies
7e5395575a Fix missing ggml link in cmake for examples/* on w64-mingw32 (#542) 2023-03-27 07:55:26 +03:00
Erik Scholz
34c1072e49 ci: add debug build to sanitizer build matrix (#527) master-34c1072 2023-03-26 15:48:40 +00:00
Stephan Walter
939ad2d3a5 Fix undefined variables in debug build, remove unused variables (#531) master-939ad2d 2023-03-26 15:34:02 +00:00
Juan Calderon-Perez
8c2ec5e21d Add support for linux/arm64 platform during Docker Builds (#514) 
* Add support for linux/arm64 platform

* Add platform to versioned builds
master-8c2ec5e 		2023-03-26 14:48:42 +00:00
Stephan Walter
b391579db9 Update README and comments for standalone perplexity tool (#525) master-b391579 2023-03-26 16:14:01 +03:00
anzz1
7a87d31f4f [main] fix infinite generation (-n == -1) (#523) master-7a87d31 2023-03-26 16:06:10 +03:00
Georgi Gerganov
348d6926ee Add logo to README.md 2023-03-26 10:20:49 +03:00
Harald Fernengel
33e35b8fe8 Exit from interactive mode if input stream is bad (#491) 
Allow exiting the interactive prompt also with CTRL-D on Unix and CTRL-Z
on Windows.
master-33e35b8 		2023-03-26 08:25:46 +03:00
anzz1
19726169b3 CI: Run other sanitizer builds even if one fails (#511) 
applies only to sanitizer builds so they wont be cancelled
master-1972616 		2023-03-26 00:13:28 +02:00
jp-x-g
f732695cd5 Clarify console output in convert-pth-to-ggml.py (#512) 
"Processing part 1 of 3" instead of "Processing part 0"
 2023-03-25 23:53:55 +02:00
anzz1
2f7bf7dd7c CMake / CI additions (#497) 
* CMake: Add AVX512 option

* CI: Add AVX/AVX512 builds (Windows)
(AVX512 tests can only be run when the worker happens to support it, building works anyway)

* CMake: Fix sanitizer linkage ( merged #468 )

* CI: Add sanitizer builds (Ubuntu)

* CI: Fix release tagging
(change @zendesk/action-create-release to @anzz1/action-create-release until upstream PR Added commitish as input zendesk/action-create-release#32 is merged)
master-2f7bf7d 		2023-03-25 23:38:11 +02:00
anzz1
34ab526843 (Windows) Set console to UTF-8 on init (#420) 
Sets console codepage to 65001 (CP_UTF8) on start for both input and output, should fix problems with UTF-8 characters.
master-34ab526 		2023-03-25 22:29:22 +02:00
Georgi Gerganov
c2b25b6912 Fix colors enabling on WIN32 master-c2b25b6 2023-03-25 21:53:39 +02:00
Georgi Gerganov
79b2b266db If n_predict == -1, generate forever 2023-03-25 21:51:41 +02:00
Georgi Gerganov
e2d490dafd Inifinite generation via context swapping (#71) 2023-03-25 21:36:22 +02:00
Georgi Gerganov
03f7e33560 Cleanup STL headers + fix embedding examples + minor stuff master-03f7e33 2023-03-25 20:51:14 +02:00
Georgi Gerganov
55ad42af84 Move chat scripts into "./examples" 2023-03-25 20:37:09 +02:00
slaren
459e93cce0 Add AVX2 implementation of dequantize_row_q4_1 (#505) master-459e93c 2023-03-25 20:31:48 +02:00
Georgi Gerganov
a316a425d0 Overhaul the examples structure 
- main -> examples
- utils -> examples (renamed to "common")
- quantize -> examples
- separate tools for "perplexity" and "embedding"

Hope I didn't break something !
master-a316a42 		2023-03-25 20:26:40 +02:00
Georgi Gerganov
ecbe466a36 Retire the ggml_mul_mat() branch for transposed src0 (#500) 
* Retire the ggml_mul_mat() for transposed src0

- It can always be made contiguous with ggml_cpy()
- The code is now simplified
- The results are deterministic in respect to num threads

* SIMD-ify dequantize_row_q4_0() for ARM_NEON (#502)

* Attempt to SIMD-ify dequantize_row_q4_0() for ARM_NEON

* Fix dequantization - forgot to interleave the quants
master-ecbe466 		2023-03-25 19:47:21 +02:00
Georgi Gerganov
502a400192 Disable prompt verbosity by default and add option to enable (#480) master-502a400 2023-03-25 17:17:16 +02:00
slaren
09aecbf628 Add AVX2 implementation of dequantize_row_q4_0 (#467) master-09aecbf 2023-03-25 17:06:49 +02:00
Georgi Gerganov
4640eff23d Don't interefe with BLAS for large prompts by running only 1 thread master-4640eff 2023-03-25 17:03:10 +02:00
Georgi Gerganov
ab77d76312 Add longer DAN prompt for testing big batch numbers 2023-03-25 16:49:09 +02:00
slaren
29b7baab67 Add timings for the prompt evaluation (#478) master-29b7baa 2023-03-25 16:34:23 +02:00