squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[V1] Fix non-cudagraph op name (#10166)

Browse Source 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
This commit is contained in:
Woosuk Kwon 2024-11-08 10:23:04 -08:00 committed by GitHub
parent 6b30471586
commit b5815c8413
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/worker/gpu_model_runner.py
View File

@ -411,7 +411,7 @@ class GPUModelRunner:
set_compilation_config( set_compilation_config(
CompilationConfig( CompilationConfig(
use_cudagraph=True, use_cudagraph=True,
non_cudagraph_ops=["vllm.unified_flash_attention"], non_cudagraph_ops=["vllm.unified_v1_flash_attention"],
use_inductor=True, use_inductor=True,
)) ))