squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

bugfix: fix the bug that stream generate not work (#2756)

Browse Source
This commit is contained in:
Zhao Yingzhuo 2024-11-09 18:09:48 +08:00 committed by GitHub
parent 8a4358ecb5
commit 9e37266420
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/entrypoints/api_server.py
View File

@ -66,7 +66,7 @@ async def generate(request: Request) -> Response:
prompt + output.text for output in request_output.outputs prompt + output.text for output in request_output.outputs
] ]
ret = {"text": text_outputs} ret = {"text": text_outputs}
yield (json.dumps(ret) + "\0").encode("utf-8") yield (json.dumps(ret) + "\n").encode("utf-8")
if stream: if stream:
return StreamingResponse(stream_results()) return StreamingResponse(stream_results())