squall/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Perf] Reduce peak memory usage of llama (#10339)

Browse Source 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
This commit is contained in:
Murali Andoorveedu 2024-11-14 16:38:20 -08:00 committed by GitHub
parent 4a18fd14ba
commit b2e0ad3b59
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/model_executor/models/llama.py
View File

@ -90,8 +90,8 @@ class LlamaMLP(nn.Module):
self.act_fn = SiluAndMul() self.act_fn = SiluAndMul()
def forward(self, x): def forward(self, x):
gate_up, _ = self.gate_up_proj(x) x, _ = self.gate_up_proj(x)
x = self.act_fn(gate_up) x = self.act_fn(x)
x, _ = self.down_proj(x) x, _ = self.down_proj(x)
return x return x