squall/flash-attention
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add newlines to README

Browse Source
This commit is contained in:
Tri Dao 2023-08-24 23:53:15 -07:00
parent 9e5e8bc91e
commit 45ba93cd96
1 changed files with 22 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

42
README.md
View File

@ -141,26 +141,28 @@ flash_attn_func(q, k, v, dropout_p=0.0, softmax_scale=None, causal=False)
If seqlen_q != seqlen_k and causal=True, the causal mask is aligned to the If seqlen_q != seqlen_k and causal=True, the causal mask is aligned to the
bottom right corner of the attention matrix, instead of the top-left corner. bottom right corner of the attention matrix, instead of the top-left corner.
For example, if seqlen_q = 2 and seqlen_k = 5, the causal mask (1 = keep, 0 = masked out) is: For example, if seqlen_q = 2 and seqlen_k = 5, the causal mask (1 = keep, 0 =
v2.0: masked out) is:
1 0 0 0 0 v2.0:
1 1 0 0 0 1 0 0 0 0
v2.1: 1 1 0 0 0
1 1 1 1 0 v2.1:
1 1 1 1 1 1 1 1 1 0
If seqlen_q = 5 and seqlen_k = 2, the causal mask is: 1 1 1 1 1
v2.0:
1 0 If seqlen_q = 5 and seqlen_k = 2, the causal mask is:
1 1 v2.0:
1 1 1 0
1 1 1 1
1 1 1 1
v2.1: 1 1
0 0 1 1
0 0 v2.1:
0 0 0 0
1 0 0 0
1 1 0 0
1 0
1 1
If the row of the mask is all zero, the output will be zero. If the row of the mask is all zero, the output will be zero.
## Performance ## Performance