4 Commits

Author	SHA1	Message	Date
jayhshah	c92ca63268	FA3 FP8 qkv descales + restore max offset for h128 causal + added sync for producer WG (#1173)	2024-08-25 12:18:04 -07:00
Tri Dao	bafe253042	[FA3] Bwd	2024-08-01 01:57:06 -07:00
jayhshah	5018ac6ac5	Fp8 kernel with "in-kernel" transpose of V in producer (#1100) ... * base version * restructure pipelines, add special fp8 epilogue * add variants * add fp8 causal and modify dynamic tile scheduler * better causal schedule * maintain two schedules for non causal and causal * removing macros * fix regression * clean up unneeded methods and variants * fix mistake with NumProducerThreads * base version * restructure pipelines, add special fp8 epilogue * add variants * add fp8 causal and modify dynamic tile scheduler * better causal schedule * maintain two schedules for non causal and causal * removing macros * fix regression * clean up unneeded methods and variants * fix mistake with NumProducerThreads * use seqlen traits * add fp8 .cu files and benchmark script * fix merge issue * fix merge issue * fix merge issue * remove duplicate code * fix regression with varseqlen * move varseqlen init in constexpr * fix test script * more constexpr on varseqlen and add max offset * add back test cases	2024-07-30 14:14:14 -07:00
Tri Dao	7f67966cc7	FA3 initial code release	2024-07-11 09:53:36 -07:00