squall/cutlass
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
06eb90cc0d
cutlass/examples/42_fused_multi_head_attention/gemm
History
dan_the_3rd 4db6a6140e
ex42: Fused MHA imported from xFormers (#662) 
* ex42: Fused MHA imported from xFormers

* Remove std:: references

* Support K>128 in the example

* Support causal option

* Support different head size for V, and different seqlength for KV

* Update FLOPS counter

* Remove bit_cast

* fix build: Replace M_LOG2E

* Add doc

* Revert "Remove bit_cast"

This reverts commit 9662fa86bb7c57c1a015ac0bf52cb52940fbbf80.

* Explicit casts to int32_t for windows build

Co-authored-by: danthe3rd <danthe3rd>
2022-10-17 10:49:33 -04:00
..
custom_mma_base.h ex42: Fused MHA imported from xFormers (#662) 2022-10-17 10:49:33 -04:00
custom_mma_multistage.h ex42: Fused MHA imported from xFormers (#662) 2022-10-17 10:49:33 -04:00
custom_mma_pipelined.h ex42: Fused MHA imported from xFormers (#662) 2022-10-17 10:49:33 -04:00
custom_mma.h ex42: Fused MHA imported from xFormers (#662) 2022-10-17 10:49:33 -04:00