squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Docs] [Spec decode] Fix docs error in code example (#5427)

Browse Source
This commit is contained in:
Cade Daniel 2024-06-11 10:31:56 -07:00 committed by GitHub
parent 9fde251bf0
commit 89ec06c33b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/models/spec_decode.rst
View File

@ -17,6 +17,7 @@ Speculating with a draft model
The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time. The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time.
.. code-block:: python .. code-block:: python
from vllm import LLM, SamplingParams from vllm import LLM, SamplingParams
prompts = [ prompts = [
@ -45,6 +46,7 @@ The following code configures vLLM to use speculative decoding where proposals a
matching n-grams in the prompt. For more information read `this thread. <https://x.com/joao_gante/status/1747322413006643259>`_ matching n-grams in the prompt. For more information read `this thread. <https://x.com/joao_gante/status/1747322413006643259>`_
.. code-block:: python .. code-block:: python
from vllm import LLM, SamplingParams from vllm import LLM, SamplingParams
prompts = [ prompts = [