6 Commits

Author	SHA1	Message	Date
Tri Dao	e78d509c64	[WIP] Support all head dimensions up to 128 in the Triton bwd ... WIP because there seems to be some race conditions for head dimensions other than 16, 32, 64, 128.	2022-10-31 00:46:22 -07:00
Tri Dao	008951f1d9	Support all head dimensions up to 128 in the Triton fwd	2022-10-30 22:10:48 -07:00
Tri Dao	b910bf14c1	Support arbitrary seqlens (both q & k) in Triton bwd	2022-10-30 21:50:53 -07:00
Tri Dao	dc55469355	Support arbitrary seqlen_k in Triton bwd	2022-10-30 21:26:26 -07:00
Tri Dao	d11341fd1a	Fix Triton fwd to support seqlen not multiples of 128	2022-10-30 19:05:47 -07:00
Tri Dao	b0c0db81f6	Implement FlashAttention in Triton	2022-10-30 18:09:11 -07:00