5 Commits

Author	SHA1	Message	Date
Tri Dao	f1e01c27ba	[Gen] Pass qkv_stride to ft_attention kernel for batched generation	2023-01-15 15:20:01 -08:00
Tri Dao	7c2191542a	[Gen] Make generation work with Tensor Parallel	2023-01-15 11:34:27 -08:00
Tri Dao	be1afaa276	[Gen, FT] Use fp32 accum for FMA	2023-01-03 22:09:22 -08:00
Tri Dao	f266fc7262	[Gen, FT] Use tlength instead of params.timestep for rotary	2023-01-03 17:46:55 -08:00
Tri Dao	a01d1213d7	[Gen] Add kernel from FasterTransformer for benchmarking	2023-01-03 17:37:43 -08:00