squall/flash-attention
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Docs] Mention FasterTransformer integration

Browse Source
This commit is contained in:
Tri Dao 2022-12-05 00:34:09 -08:00
parent 4a6eaa9f27
commit a84d07283c
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
usage.md
View File

@ -50,6 +50,11 @@ yields the fastest BERT training on cloud instances in MLPerf training 2.0 (June
uses FlashAttention as part of their approach to speed up Transformer
inference (up to 5.3x on BERT).
- Nvidia's [FasterTransformer](https://github.com/NVIDIA/FasterTransformer) is a
state-of-the-art Transformer inference library. As of version
[5.2](https://github.com/NVIDIA/FasterTransformer/commit/b672f49e256ba7a2d4fc9691d270b60b7fc1a2ff),
FlashAttention is used as a component of FasterTransformer to speed up GPT inference.
- [Kernl](https://github.com/ELS-RD/kernl) is a library for fast Transformer
inference. They use FlashAttention as part of their
[approach](https://twitter.com/pommedeterre33/status/1585284221014245377) to