5 Commits

Author	SHA1	Message	Date
Tri Dao	0938298e4c	[Gen] Adjust shape of kv_cache when using FT	2023-01-07 17:27:54 -08:00
Tri Dao	e02fd588aa	[Gen] Implement top-k and top-p sampling	2023-01-07 17:00:02 -08:00
Tri Dao	11be742aa3	[Gen] Test generation with rotary embedding	2023-01-07 14:37:54 -08:00
Tri Dao	a668890fcd	[Gen] Add option to run generation with FT attention kernel	2023-01-03 22:10:31 -08:00
Tri Dao	63670fd84a	Implement generation for GPT	2022-12-27 21:01:50 -08:00