squall/flash-attention
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add gpt-neox adoption

Browse Source
This commit is contained in:
Quentin Anthony 2022-12-28 18:33:58 -05:00 committed by GitHub
parent a6ec1782dc
commit d2a69a55e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
usage.md
View File

@ -24,6 +24,8 @@ PR or email us. We'd very much like to hear from you!
- MosaicML [Composer](https://github.com/mosaicml/composer)
[library](https://www.mosaicml.com/blog/gpt-3-quality-for-500k). Composer is a
library for efficient neural network training.
- EleutherAI's [GPT-NeoX](https://github.com/EleutherAI/gpt-neox/pull/725). This is a research library for training large language transformer models at scale based on NVIDIA's Megatron-LM and Microsoft's DeepSpeed.
## MLPerf benchmarks