squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add Mistral to quantization model list (#1278)

Browse Source
This commit is contained in:
amaleshvemula 2023-10-11 09:26:24 +02:00 committed by GitHub
parent 980dd4a2c4
commit ee8217e5be
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/model_executor/model_loader.py
View File

@ -35,6 +35,7 @@ _MODEL_REGISTRY = {
# FIXME(woosuk): Remove this once all models support quantization. # FIXME(woosuk): Remove this once all models support quantization.
_MODEL_CLASSES_SUPPORT_QUANTIZATION = [ _MODEL_CLASSES_SUPPORT_QUANTIZATION = [
LlamaForCausalLM, LlamaForCausalLM,
MistralForCausalLM,
] ]