vllm/requirements.txt

ninja  # For faster builds.
psutil
ray >= 2.5.1
sentencepiece  # Required for LLaMA tokenizer.
numpy
torch == 2.1.2
transformers >= 4.37.0 # Required for Qwen2
xformers == 0.0.23.post1  # Required for CUDA 12.1.
fastapi
uvicorn[standard]
pydantic >= 2.0  # Required for OpenAI server.
aioprometheus[starlette]
pynvml == 11.5.0
Specify python package dependencies in requirements.txt (#78) 2023-05-08 07:30:43 +08:00			`ninja # For faster builds.`
			`psutil`
Ray placement group support (#397) 2023-07-20 13:49:31 +08:00			`ray >= 2.5.1`
Specify python package dependencies in requirements.txt (#78) 2023-05-08 07:30:43 +08:00			`sentencepiece # Required for LLaMA tokenizer.`
			`numpy`
Pin PyTorch & xformers versions (#2155) 2023-12-17 17:46:54 +08:00			`torch == 2.1.2`
Add qwen2 (#2495) 2024-01-23 06:34:21 +08:00			`transformers >= 4.37.0 # Required for Qwen2`
[Minor] Fix xformers version (#2158) 2023-12-17 18:28:02 +08:00			`xformers == 0.0.23.post1 # Required for CUDA 12.1.`
Specify python package dependencies in requirements.txt (#78) 2023-05-08 07:30:43 +08:00			`fastapi`
Use standard extras for uvicorn (#1166) 2023-09-28 08:41:36 +08:00			`uvicorn[standard]`
migrate pydantic from v1 to v2 (#2531) 2024-01-22 08:05:56 +08:00			`pydantic >= 2.0 # Required for OpenAI server.`
Add Production Metrics in Prometheus format (#1890) 2023-12-03 08:37:44 +08:00			`aioprometheus[starlette]`
Implement custom all reduce kernels (#2192) 2024-01-28 04:46:35 +08:00			`pynvml == 11.5.0`