squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Correct alignment in the seq_len diagram. (#5592)

Browse Source 
Co-authored-by: Liqian Chen <liqian.chen@deeplang.ai>
This commit is contained in:
Charles Riggins 2024-06-18 00:05:33 +08:00 committed by GitHub
parent 9333fb8eb9
commit 9e74d9d003
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/attention/backends/flash_attn.py
View File

@ -83,7 +83,7 @@ class FlashAttentionMetadata(AttentionMetadata):
# |---------------- N iteration ---------------------| # |---------------- N iteration ---------------------|
# |- tokenA -|......................|-- newTokens ---| # |- tokenA -|......................|-- newTokens ---|
# |---------- context_len ----------| # |---------- context_len ----------|
# |-------------------- seq_len ----------------------| # |-------------------- seq_len ---------------------|
# |-- query_len ---| # |-- query_len ---|
# Maximum query length in the batch. None for decoding. # Maximum query length in the batch. None for decoding.