Tri Dao e78d509c64 [WIP] Support all head dimensions up to 128 in the Triton bwd ... WIP because there seems to be some race conditions for head dimensions other than 16, 32, 64, 128.		2022-10-31 00:46:22 -07:00
..
utils	Move benchmark utils, support AMP	2022-10-23 12:50:00 -07:00
__init__.py	Add missing __init__.py	2022-07-03 02:04:55 -04:00
bert_padding.py	remove numpy dependency	2022-10-06 19:17:15 +02:00
flash_attention.py	Relax assert to allow both bf16 and fp16	2022-09-11 12:09:43 -07:00
flash_attn_interface.py	Support all head dims that are multiples of 8, up to 128	2022-10-24 16:04:21 -07:00
flash_attn_triton_og.py	Implement FlashAttention in Triton	2022-10-30 18:09:11 -07:00
flash_attn_triton.py	[WIP] Support all head dimensions up to 128 in the Triton bwd	2022-10-31 00:46:22 -07:00
flash_blocksparse_attention.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
flash_blocksparse_attn_interface.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
fused_softmax.py	Add Megatron attention implementation for benchmarking	2022-10-23 23:04:16 -07:00
rotary.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00