From b0ed0a73fdf1b905af84a5e4561201b773ee625c Mon Sep 17 00:00:00 2001
From: Tri Dao <tridpq@gmail.com>
Date: Mon, 14 Nov 2022 10:01:16 -0800
Subject: [PATCH] Mention DeepSpeed inference in usage.md

---
 usage.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/usage.md b/usage.md
index 7fdc454..e6651b3 100644
--- a/usage.md
+++ b/usage.md
@@ -15,6 +15,9 @@ PR or email us. We'd very much like to hear from you!
   [On-going](https://github.com/huggingface/transformers/pull/18439), blogpost
   coming soon.
 
+- Microsoft's [DeepSpeed](https://github.com/microsoft/DeepSpeed):
+  FlashAttention is [integrated](https://github.com/microsoft/DeepSpeed/blob/ec13da6ba7cabc44bb4745a64a208b8580792954/deepspeed/ops/transformer/inference/triton_ops.py) into DeepSpeed's inference engine.
+
 - MosaicML [Composer](https://github.com/mosaicml/composer)
   [library](https://www.mosaicml.com/blog/gpt-3-quality-for-500k). Composer is a
   library for efficient neural network training.