Jongwook Choi 12e8c12d2b Disable CUDA peer access as a workaround for multi-gpu inference bug (#1261) ... When CUDA peer access is enabled, multi-gpu inference will produce garbage output. This is a known bug of llama.cpp (or nvidia). Until the upstream bug is fixed, we can disable CUDA peer access temporarily to ensure correct output. See #961.		2023-11-24 14:05:57 -05:00
..
llama.cpp	Disable CUDA peer access as a workaround for multi-gpu inference bug (#1261)	2023-11-24 14:05:57 -05:00
falcon.go	starcoder	2023-10-02 19:56:51 -07:00
ggml.go	ggufv3	2023-10-23 09:35:49 -07:00
gguf.go	fix: gguf int type	2023-11-22 11:40:30 -08:00
llama.go	only set main_gpu if value > 0 is provided	2023-11-20 19:54:04 -05:00
llm.go	recent llama.cpp update added kernels for fp32, q5_0, and q5_1	2023-11-20 13:44:31 -08:00
starcoder.go	starcoder	2023-10-02 19:56:51 -07:00
utils.go	partial decode ggml bin for more info	2023-08-10 09:23:10 -07:00