5 Commits

Author	SHA1	Message	Date
Tri Dao	88173a1aaf	[FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP	2023-01-17 18:12:27 -08:00
Tri Dao	780e8eeabb	[ViT] Support timm checkpoint, add tests	2023-01-16 01:20:34 -08:00
Tri Dao	ef085cfcda	[ViT] Fix extra norm_0, use new LN order in Block	2023-01-15 22:58:56 -08:00
Tri Dao	1feb94265c	[ViT] Use dropout_add_ln for the 1st layer norm	2022-11-23 12:48:56 -08:00
Tri Dao	2e33fc8e36	Add GPT and ViT models	2022-11-13 22:30:23 -08:00