squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Doc]: Update tensorizer docs to include vllm[tensorizer] (#7889)

Browse Source 
Co-authored-by: Kaunil Dhruv <dhruv.kaunil@gmail.com>
This commit is contained in:
Seth Kimmel 2024-10-22 15:43:25 -07:00 committed by GitHub
parent b17046e298
commit 208cb34c81
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/source/serving/tensorizer.rst
View File

@ -10,3 +10,6 @@ shorter Pod startup times and CPU memory usage. Tensor encryption is also suppor
For more information on CoreWeave's Tensorizer, please refer to For more information on CoreWeave's Tensorizer, please refer to
`CoreWeave's Tensorizer documentation <https://github.com/coreweave/tensorizer>`_. For more information on serializing a vLLM model, as well a general usage guide to using Tensorizer with vLLM, see `CoreWeave's Tensorizer documentation <https://github.com/coreweave/tensorizer>`_. For more information on serializing a vLLM model, as well a general usage guide to using Tensorizer with vLLM, see
the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_. the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_.
.. note::
Note that to use this feature you will need to install `tensorizer` by running `pip install vllm[tensorizer]`.