squall/picotron
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
68 Commits 1 Branch 0 Tags 3.1 MiB
Python 98%
Shell 2%
8e36bbe032
Go to file
ferdinand.mom 8e36bbe032 fix multi-node training by using global rank instead of local rank to init process_group
2024-11-03 00:14:14 +00:00
src fix DP integation within PP (1f1b) 2024-11-01 20:08:48 +00:00
template fix multi-node training by using global rank instead of local rank to init process_group 2024-11-03 00:14:14 +00:00
.gitignore tesnsor parallel, will clean later 2024-10-18 05:13:44 +00:00
convert_hf_to_picotron.py various fix (modeling, dataloader, cpu load) 2024-10-18 14:33:46 +00:00
convert_picotron_to_hf.py refactor organisation 2024-10-10 15:12:14 +00:00
create_config.py add fuse adam 2024-11-02 01:38:14 +00:00
generate.py various fix (modeling, dataloader, cpu load) 2024-10-18 14:33:46 +00:00
model.py fix spliting input twice for context parallel (done in dataloader) 2024-10-30 15:43:42 +00:00
README.md Initial commit 2024-09-18 14:01:22 +02:00
requirements.txt fix requirements to avoid drop in throughput 2024-11-02 02:18:49 +00:00
setup.py tesnsor parallel, will clean later 2024-10-18 05:13:44 +00:00
submit_slurm_jobs.py add hf token + fix multi-node training with torchrun args 2024-11-02 02:18:40 +00:00
train.py fix multi-node training by using global rank instead of local rank to init process_group 2024-11-03 00:14:14 +00:00
utils.py fix multi-node training by using global rank instead of local rank to init process_group 2024-11-03 00:14:14 +00:00

README.md

picotron