5 Commits

Author	SHA1	Message	Date
Tri Dao	c6ecd40a59	Tweak CrossEntropyLoss to take process_group in init	2022-12-27 10:47:43 -08:00
Tri Dao	b4018a5028	Implement Tensor Parallel for GPT model	2022-12-26 16:22:43 -08:00
Tri Dao	226a1b721d	Implement TensorParallel for FusedDense and FusedDenseGeluDense	2022-12-24 11:48:56 -08:00
Tri Dao	ece539abd6	Add __init__.py files to subdirectories for installation	2022-11-17 16:55:44 -08:00
Tri Dao	fb88e5e4b3	Move benchmark utils, support AMP	2022-10-23 12:50:00 -07:00