Tri Dao a4f148b6ab Fix masking of bwd when seqlen is not divisible by 128		2023-07-31 17:46:34 -07:00
..
layers	[Rotary] Implement GPT-J style (interleaved) rotary	2023-03-14 14:35:53 -07:00
losses	Tweak CrossEntropyLoss to take process_group in init	2022-12-27 10:47:43 -08:00
models	[GPT] Implement parallel LLaMa	2023-07-28 15:52:48 -10:00
modules	Implement ParallelGatedMlp (#251)	2023-07-26 12:14:15 -07:00
ops	[LayerNorm] Add test for randomness	2023-07-23 12:31:55 -10:00
test_flash_attn.py	Fix masking of bwd when seqlen is not divisible by 128	2023-07-31 17:46:34 -07:00
test_rotary.py	Add MLP, MHA, Block, Embedding modules	2022-11-13 22:06:44 -08:00