diff --git a/README.md b/README.md index ca3e318..3330f2c 100644 --- a/README.md +++ b/README.md @@ -40,14 +40,14 @@ Our graphs show sequence lengths between 128 and 4096 (when standard attention r #### Speedup -![FlashAttention speedup](images/flashattn_speedup.png) +![FlashAttention speedup](images/flashattn_speedup.jpg) We generally see 2-4X speedup at sequence lengths between 128 and 4K, and we see more speedup when using dropout and masking, since we fuse the kernels. At sequence lengths that are popular with language models like 512 and 1K, we see speedups up to 4X when using dropout and masking. #### Memory -![FlashAttention memory](images/flashattn_memory.png) +![FlashAttention memory](images/flashattn_memory.jpg) We show memory savings in this graph (note that memory footprint is the same no matter if you use dropout or masking). Memory savings are proportional to sequence length -- since standard attention has memory quadratic in sequence length, whereas FlashAttention has memory linear in sequence length. diff --git a/images/flashattn_memory.jpg b/images/flashattn_memory.jpg new file mode 100644 index 0000000..d0b6b19 Binary files /dev/null and b/images/flashattn_memory.jpg differ diff --git a/images/flashattn_memory.png b/images/flashattn_memory.png deleted file mode 100644 index 42a82ec..0000000 Binary files a/images/flashattn_memory.png and /dev/null differ diff --git a/images/flashattn_speedup.jpg b/images/flashattn_speedup.jpg new file mode 100644 index 0000000..79fb4d5 Binary files /dev/null and b/images/flashattn_speedup.jpg differ diff --git a/images/flashattn_speedup.png b/images/flashattn_speedup.png deleted file mode 100644 index 3b1a895..0000000 Binary files a/images/flashattn_speedup.png and /dev/null differ