squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Connect engine healthcheck to openai server (#3260)

Browse Source
This commit is contained in:
Nick Hill 2024-03-07 16:38:12 -08:00 committed by GitHub
parent b35cc93420
commit d2339d6840
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/entrypoints/openai/api_server.py
View File

@ -160,6 +160,7 @@ async def validation_exception_handler(_, exc):
@app.get("/health")
async def health() -> Response:
"""Health check."""
await openai_serving_chat.engine.check_health()
return Response(status_code=200)