From cb1362a8892d444294117b848547c76b80d6dc65 Mon Sep 17 00:00:00 2001 From: Woosuk Kwon Date: Tue, 23 Jul 2024 08:18:15 -0700 Subject: [PATCH] [Docs] Announce llama3.1 support (#6688) --- README.md | 1 + docs/source/models/supported_models.rst | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 725c69b3..8e508195 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ Register now [here](https://lu.ma/lp0gyjqr) and be part of the event! --- *Latest News* 🔥 +- [2024/07] In partnership with Meta, vLLM officially supports Llama 3.1 with FP8 quantization and pipeline parallelism! Please check out our blog post [here](https://blog.vllm.ai/2024/07/23/llama31.html). - [2024/06] We hosted [the fourth vLLM meetup](https://lu.ma/agivllm) with Cloudflare and BentoML! Please find the meetup slides [here](https://docs.google.com/presentation/d/1iJ8o7V2bQEi0BFEljLTwc5G1S10_Rhv3beed5oB0NJ4/edit?usp=sharing). - [2024/04] We hosted [the third vLLM meetup](https://robloxandvllmmeetup2024.splashthat.com/) with Roblox! Please find the meetup slides [here](https://docs.google.com/presentation/d/1A--47JAK4BJ39t954HyTkvtfwn0fkqtsL8NGFuslReM/edit?usp=sharing). - [2024/01] We hosted [the second vLLM meetup](https://lu.ma/ygxbpzhl) with IBM! Please find the meetup slides [here](https://docs.google.com/presentation/d/12mI2sKABnUw5RBWXDYY-HtHth4iMSNcEoQ10jDQbxgA/edit?usp=sharing). diff --git a/docs/source/models/supported_models.rst b/docs/source/models/supported_models.rst index f49e4537..068c00da 100644 --- a/docs/source/models/supported_models.rst +++ b/docs/source/models/supported_models.rst @@ -94,8 +94,8 @@ Decoder-only Language Models - :code:`ai21labs/Jamba-v0.1`, etc. - ✅︎ * - :code:`LlamaForCausalLM` - - LLaMA, Llama 2, Meta Llama 3, Vicuna, Alpaca, Yi - - :code:`meta-llama/Meta-Llama-3-8B-Instruct`, :code:`meta-llama/Meta-Llama-3-70B-Instruct`, :code:`meta-llama/Llama-2-13b-hf`, :code:`meta-llama/Llama-2-70b-hf`, :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`01-ai/Yi-6B`, :code:`01-ai/Yi-34B`, etc. + - Llama 3.1, Llama 3, Llama 2, LLaMA, Yi + - :code:`meta-llama/Meta-Llama-3.1-405B-Instruct`, :code:`meta-llama/Meta-Llama-3.1-70B`, :code:`meta-llama/Meta-Llama-3-70B-Instruct`, :code:`meta-llama/Llama-2-70b-hf`, :code:`01-ai/Yi-34B`, etc. - ✅︎ * - :code:`MiniCPMForCausalLM` - MiniCPM