picotron/requirements.txt

torch==2.1.0
triton==2.1.0
numpy==1.26.4
datasets==2.19.1
transformers==4.47.0
flash-attn==2.5.0
wandb
huggingface_hub[hf_transfer]
fix requirements to avoid drop in throughput 2024-11-04 22:33:07 +08:00			`torch==2.1.0`
breaking: socketStartConnect: Connect to <ip address> failed : Software caused connection abort 2024-09-24 05:14:48 +08:00			`triton==2.1.0`
			`numpy==1.26.4`
			`datasets==2.19.1`
download safetensors when creating config time. If we do it in training, barrier() may tiemout while waiting for download 2024-12-17 23:41:00 +08:00			`transformers==4.47.0`
fix requirements to avoid drop in throughput 2024-11-04 22:33:07 +08:00			`flash-attn==2.5.0`
use hf_transfer which improve download time by 3 2024-12-18 22:51:14 +08:00			`wandb`
			`huggingface_hub[hf_transfer]`