flash-attention/csrc/layer_norm/README.md

This CUDA extensions implements fused dropout + residual + LayerNorm, based on Apex's FastLayerNorm. We add dropout and residual, and make it work for both pre-norm and post-norm architecture.

cd csrc/layer_norm && pip install .