Tri Dao 75e334d407 [MLP] Add ParallelMLP		2023-07-22 23:45:51 -07:00
..
layers	rotary: update cos/sin cache when switching from inference mode	2023-07-08 12:01:07 +02:00
losses	Tweak CrossEntropyLoss to take process_group in init	2022-12-27 10:47:43 -08:00
models	[MLP] Add ParallelMLP	2023-07-22 23:45:51 -07:00
modules	[MLP] Add ParallelMLP	2023-07-22 23:45:51 -07:00
ops	[LayerNorm] Make sure memory addresses are aligned to 16 bytes	2023-07-04 14:53:12 -07:00
utils	[Gen] Minor tweak to allocate_inference_cache	2023-04-21 11:56:47 -07:00
__init__.py	FlashAttention-2 release	2023-07-17 06:21:34 -07:00
bert_padding.py	remove numpy dependency	2022-10-06 19:17:15 +02:00
flash_attn_interface.py	Make sure dout is contiguous	2023-07-17 21:54:44 -07:00
flash_attn_triton_og.py	Implement FlashAttention in Triton	2022-10-30 18:09:11 -07:00
flash_attn_triton.py	[Triton] Fix benchmark_causal, mention Triton version	2023-03-22 00:51:16 -07:00
flash_blocksparse_attention.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
flash_blocksparse_attn_interface.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
fused_softmax.py	Add Megatron attention implementation for benchmarking	2022-10-23 23:04:16 -07:00