ferdinand.mom
|
abd1edf9f9
|
all_reduce loss across pp/dp ranks + base_parallel
|
2024-10-18 15:51:17 +00:00 |
|
ferdinand.mom
|
1ebd3de5be
|
Merge DDP + TP from @zzhhjjj
|
2024-10-18 15:05:01 +00:00 |
|
ferdinand.mom
|
d0d6d8994f
|
use global pgm for ddp
|
2024-10-18 14:59:26 +00:00 |
|
ferdinand.mom
|
134d48b658
|
remove merged qkv
|
2024-10-18 14:59:04 +00:00 |
|
zzhhjjj
|
7377238741
|
tesnsor parallel, will clean later
|
2024-10-18 05:13:44 +00:00 |
|
zzhhjjj
|
54ad77e055
|
Merge branch 'main' into ddp-merge
|
2024-10-16 19:13:48 +00:00 |
|
zzhhjjj
|
24ff8d05fd
|
add DDP
|
2024-10-16 16:48:55 +00:00 |
|
zzhhjjj
|
5139a32211
|
repo structure change
|
2024-10-16 16:44:39 +00:00 |
|