squall/picotron
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Ferdinand Mom 2024-12-20 13:56:34 +01:00 committed by GitHub
parent bf03420686
commit df3ae8a5f0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -6,6 +6,12 @@ In the spirit of [NanoGPT](https://github.com/karpathy/nanoGPT), we created Pico
- Performance is not the best but still under active development. We observed 38% MFU on a LLaMA-2-7B model using 64 H100 GPUs and nearly 50% MFU on the SmolLM-1.7B model with 8 H100 GPUs. Benchmarks will come soon
# Tutorial videos
- A step by step tutorial on how to build Picotron distributed training framework form scratch:
- [Picotron tutorial (playlist)](https://www.youtube.com/playlist?list=PL-_armZiJvAnhcRr6yTJ0__f3Oi-LLi9S) 🎬
- [Picotron tutorial (codebase)](https://github.com/huggingface/picotron_tutorial) 👷🏻‍♂️
# Install
```