Commit Graph

4 Commits

Author SHA1 Message Date
Tri Dao
4cab4de5ea [TP] Put parallel embeddings in separate modules 2023-01-02 08:47:48 -08:00
Tri Dao
78225c5366 Implement Tensor Parallel for GPT2Embeddings 2022-12-25 14:29:53 -08:00
Tri Dao
5fb6df0e04 Implement BERT 2022-12-18 21:47:27 -08:00
Tri Dao
d4b320b31f Add MLP, MHA, Block, Embedding modules 2022-11-13 22:06:44 -08:00