squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[MISC] Remove FP8 warning (#5472)

Browse Source 
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
This commit is contained in:
Cody Yu 2024-06-13 11:22:30 -07:00 committed by GitHub
parent 85657b5607
commit 30299a41fa
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/config.py
View File

@ -212,7 +212,7 @@ class ModelConfig:
f"{self.quantization} quantization is currently not " f"{self.quantization} quantization is currently not "
f"supported in ROCm.") f"supported in ROCm.")
if (self.quantization if (self.quantization
not in ["marlin", "gptq_marlin_24", "gptq_marlin"]): not in ("fp8", "marlin", "gptq_marlin_24", "gptq_marlin")):
logger.warning( logger.warning(
"%s quantization is not fully " "%s quantization is not fully "
"optimized yet. The speed can be slower than " "optimized yet. The speed can be slower than "