squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
e38074b1e6
vllm/benchmarks
History
Zhuohan Li 1a956e136b
Fix various issues of async servers (#135)
2023-06-05 23:44:50 +08:00
..
benchmark_async_llm_server.py Fix various issues of async servers (#135) 2023-06-05 23:44:50 +08:00
benchmark_latency.py Add docstrings for LLM (#137) 2023-06-04 12:52:41 -07:00
benchmark_text_completion.py Add throughput benchmarking script (#133) 2023-05-28 03:20:05 -07:00
benchmark_throughput.py Add docstrings for LLM (#137) 2023-06-04 12:52:41 -07:00
README.md Add throughput benchmarking script (#133) 2023-05-28 03:20:05 -07:00
trace.py Add throughput benchmarking script (#133) 2023-05-28 03:20:05 -07:00

README.md

Benchmarking CacheFlow

Downloading the ShareGPT dataset

You can download the dataset by running:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json