Tri Dao 557781933d Parallelize CUDA bwd along seqlen_k instead of seqlen_q ... This is faster since we only need to do atomic adds on dq, instead of atomic adds on both dk and dv.		2022-11-05 16:26:17 -07:00
..
utils	Move benchmark utils, support AMP	2022-10-23 12:50:00 -07:00
__init__.py	Add missing __init__.py	2022-07-03 02:04:55 -04:00
bert_padding.py	remove numpy dependency	2022-10-06 19:17:15 +02:00
flash_attention.py	Relax assert to allow both bf16 and fp16	2022-09-11 12:09:43 -07:00
flash_attn_interface.py	Parallelize CUDA bwd along seqlen_k instead of seqlen_q	2022-11-05 16:26:17 -07:00
flash_attn_triton_og.py	Implement FlashAttention in Triton	2022-10-30 18:09:11 -07:00
flash_attn_triton.py	Parallelize CUDA bwd along seqlen_k instead of seqlen_q	2022-11-05 16:26:17 -07:00
flash_blocksparse_attention.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
flash_blocksparse_attn_interface.py	Rename src -> flash_attn	2022-06-01 18:50:26 -07:00
fused_softmax.py	Add Megatron attention implementation for benchmarking	2022-10-23 23:04:16 -07:00
rotary.py	Implement rotary embedding in CUDA	2022-11-04 22:42:01 -07:00