From 4decc3c166df32d3c4b8e02c0d18c6c0786d42c2 Mon Sep 17 00:00:00 2001 From: Dan Fu Date: Fri, 27 May 2022 22:38:20 +0100 Subject: [PATCH] README typo --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index d2c553e..ca3e318 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ We display FlashAttention speedup using these parameters (similar to BERT-base): * Batch size 8 * Head dimension 64 * 12 attention heads + Our graphs show sequence lengths between 128 and 4096 (when standard attention runs out of memory on an A100), but FlashAttention can scale up to sequence length 64K. #### Speedup