Xuechen Li 0f7853c6a1 enable loading hf llama checkpoints for training (#446) ... * prelim. * add hf convertion fn. * mlp. * change name. * fix bug. * inverse permute. * change comment. * revert style changes. * fix. * add doc. * revert. * enable load safe. * fix safe load. * fix import. * fix typing-related lints. * fix ckpt loading logic. * make single gpu work. * test with parallel. * ckpt format. * enable pretrained state dict. * remove unused imports. * remove unused. * mark idea related.		2023-08-15 08:33:15 -07:00
..
layers	[Rotary] Implement GPT-J style (interleaved) rotary	2023-03-14 14:35:53 -07:00
losses	Tweak CrossEntropyLoss to take process_group in init	2022-12-27 10:47:43 -08:00
models	enable loading hf llama checkpoints for training (#446)	2023-08-15 08:33:15 -07:00
modules	Implement ParallelGatedMlp (#251)	2023-07-26 12:14:15 -07:00
ops	[LayerNorm] Add test for randomness	2023-07-23 12:31:55 -10:00
test_flash_attn.py	Update to Cutlass 3.1	2023-08-13 13:53:17 -07:00
test_rotary.py	Add MLP, MHA, Block, Embedding modules	2022-11-13 22:06:44 -08:00