Jeffrey Morgan b24e8d17b2 Increase minimum CUDA memory allocation overhead and fix minimum overhead for multi-gpu (#1896) ... * increase minimum cuda overhead and fix minimum overhead for multi-gpu * fix multi gpu overhead * limit overhead to 10% of all gpus * better wording * allocate fixed amount before layers * fixed only includes graph alloc		2024-01-10 19:08:51 -05:00
..
ext_server	Update submodule to 6efb8eb30e7025b168f3fda3ff83b9b386428ad6 (#1885)	2024-01-10 16:48:38 -05:00
generate	clean up cmake build directory when cross compiling macOS builds	2024-01-09 17:13:56 -05:00
llama.cpp@328b83de23	revert submodule back to 328b83de23b33240e28f4e74900d1d06726f5eb1	2024-01-10 18:42:39 -05:00
dynamic_shim.c	Switch windows build to fully dynamic	2024-01-02 15:36:16 -08:00
dynamic_shim.h	Refactor how we augment llama.cpp	2024-01-02 15:35:55 -08:00
ext_server_common.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
ext_server_default.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
ext_server_windows.go	fix windows build	2024-01-08 20:04:01 -05:00
ggml.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
gguf.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
llama.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
llm.go	Increase minimum CUDA memory allocation overhead and fix minimum overhead for multi-gpu (#1896)	2024-01-10 19:08:51 -05:00
shim_darwin.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
shim_ext_server_linux.go	Code shuffle to clean up the llm dir	2024-01-04 12:12:05 -08:00
shim_ext_server_windows.go	Code shuffle to clean up the llm dir	2024-01-04 12:12:05 -08:00
shim_ext_server.go	Offload layers to GPU based on new model size estimates (#1850)	2024-01-08 16:42:00 -05:00
utils.go	partial decode ggml bin for more info	2023-08-10 09:23:10 -07:00