From 3dda4f76deeee9e17ad03e8608d33e7a5fa714bf Mon Sep 17 00:00:00 2001 From: Tri Dao Date: Sun, 13 Nov 2022 16:49:11 -0800 Subject: [PATCH] Update README --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3956fa6..35250ec 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,13 @@ Paper: https://arxiv.org/abs/2205.14135 IEEE Spectrum [article](https://spectrum.ieee.org/mlperf-rankings-2022) about our submission to the MLPerf 2.0 benchmark using FlashAttention. ![FlashAttention](assets/flashattn_banner.jpg) -#### Triton implementation of FlashAttention +## Usage + +We've been very happy to see FlashAttention being widely adopted in such a short +time after its release. This [page](https://github.com/HazyResearch/flash-attention/blob/main/usage.md) +contains a partial list of places where FlashAttention is being used. + +## Triton implementation of FlashAttention Phil Tillet (OpenAI) has an experimental implementation of FlashAttention in Triton: https://github.com/openai/triton/blob/master/python/tutorials/06-fused-attention.py