squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Bugfix] Fix token padding for chameleon (#6724)

Browse Source
This commit is contained in:
Roger Wang 2024-07-24 01:05:09 -07:00 committed by GitHub
parent c882a7f5b3
commit 0a740a11ba
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/model_executor/models/chameleon.py
View File

@ -125,7 +125,8 @@ def input_processor_for_chameleon(ctx: InputContext, llm_inputs: LLMInputs):
# Appending sep token for chat mode to follow default processor
# behavior
new_prompt += tokenizer.sep_token
if new_prompt is not None:
new_prompt += tokenizer.sep_token
new_token_ids += [CHAMELEON_SEP_TOKEN_ID]
# NOTE: Create a defensive copy of the original inputs