squall/flash-attention
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7d25a4ec4f
flash-attention/csrc/flash_attn/src
History
Kirthi Shankar Sivamani 31018c5fa0 Support CUDA graph capture 
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
2023-04-12 16:53:22 -07:00
..
fmha Parallelize CUDA bwd along seqlen_k instead of seqlen_q 2022-11-05 16:26:17 -07:00
fmha_block_dgrad_fp16_kernel_loop.sm80.cu Implement cross attention 2022-07-03 17:48:12 -07:00
fmha_block_dgrad_kernel_1xN_loop.h Support all head dims that are multiples of 8, up to 128 2022-10-24 16:04:21 -07:00
fmha_block_fprop_fp16_kernel.sm80.cu Implement cross attention 2022-07-03 17:48:12 -07:00
fmha_block_fprop_kernel_1xN.h Support all head dims that are multiples of 8, up to 128 2022-10-24 16:04:21 -07:00
fmha_blockmask.h Implement cross attention 2022-07-03 17:48:12 -07:00
fmha_bwd_hdim32.cu [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_bwd_hdim64.cu Support H100 2023-03-15 14:59:02 -07:00
fmha_bwd_hdim128.cu [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_bwd_launch_template.h [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_dgrad_kernel_1xN_loop.h Support CUDA graph capture 2023-04-12 16:53:22 -07:00
fmha_fprop_kernel_1xN.h Support CUDA graph capture 2023-04-12 16:53:22 -07:00
fmha_fwd_hdim32.cu [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_fwd_hdim64.cu [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_fwd_hdim128.cu [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_fwd_launch_template.h [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00
fmha_kernel.h Speed up compilation by splitting into separate .cu files 2022-11-25 16:30:18 -08:00
fmha_utils.h Implement for bf16 2022-07-09 23:31:56 -07:00
fmha.h Support CUDA graph capture 2023-04-12 16:53:22 -07:00
philox.cuh Rework dropout to decouple forward and backward 2022-10-21 12:04:27 -07:00
static_switch.h [Compilation] Change BOOL_SWITCH to fix Windows compilation 2023-01-06 14:40:58 -08:00