squall/picotron

Go to file

ferdinand.mom 0bfc06506a small changes unrelated to dp+pp sync grad fix		2024-11-04 15:00:43 +00:00
src	some dp renaming	2024-11-04 14:48:12 +00:00
template	set num workers to 1 for now to avoid os memory error	2024-11-04 14:39:52 +00:00
.gitignore	tesnsor parallel, will clean later	2024-10-18 05:13:44 +00:00
convert_hf_to_picotron.py	various fix (modeling, dataloader, cpu load)	2024-10-18 14:33:46 +00:00
convert_picotron_to_hf.py	refactor organisation	2024-10-10 15:12:14 +00:00
create_config.py	small changes unrelated to dp+pp sync grad fix	2024-11-04 15:00:43 +00:00
generate.py	various fix (modeling, dataloader, cpu load)	2024-10-18 14:33:46 +00:00
model.py	fix spliting input twice for context parallel (done in dataloader)	2024-10-30 15:43:42 +00:00
README.md	Initial commit	2024-09-18 14:01:22 +02:00
requirements.txt	fix requirements to avoid drop in throughput	2024-11-04 14:33:07 +00:00
setup.py	tesnsor parallel, will clean later	2024-10-18 05:13:44 +00:00
submit_slurm_jobs.py	add option for HF token	2024-11-04 14:39:12 +00:00
train.py	small changes unrelated to dp+pp sync grad fix	2024-11-04 15:00:43 +00:00
utils.py	better config creation	2024-10-30 14:58:41 +00:00

README.md

picotron