Skip to content

Commit ddae26e

Browse files
authored
Merge pull request #2 from ruidazeng/patch-2
Patch 2
2 parents d164903 + 30db2bf commit ddae26e

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/source/bettertransformer/tutorials/contribute.mdx

+2-2
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ Now, make sure to fill all the necessary attributes, the list of attributes are:
112112

113113
Note that these attributes correspond to all the components that are necessary to run a Transformer Encoder module, check the figure 1 on the ["Attention Is All You Need"](https://arxiv.org/pdf/1706.03762.pdf) paper.
114114

115-
Once you filled all these attributes (sometimes the `query`, `key` and `value` layers needs to be "contigufied", check the [`modeling_encoder.py`](https://github.com/huggingface/optimum/blob/main/optimum/bettertransformer/models/encoder_models.py) file to understand more.)
115+
Once you filled all these attributes (sometimes the `query`, `key` and `value` layers needs to be "contiguified", check the [`modeling_encoder.py`](https://github.com/huggingface/optimum/blob/main/optimum/bettertransformer/models/encoder_models.py) file to understand more.)
116116

117117
Make sure also to add the lines:
118118
```python
@@ -125,7 +125,7 @@ self.validate_bettertransformer()
125125

126126
First of all, start with the line `super().forward_checker()`, this is needed so that the parent class can run all the safety checkers before.
127127

128-
After the first forward pass, the hidden states needs to be *nested* using the attention mask. Once they are nested, the attention mask is not needed anymore, therefore can be set to `None`. This is how the forward pass is built for `Bert`, these lines should remain pretty much similar accross models, but sometimes the shapes of the attention masks are different across models.
128+
After the first forward pass, the hidden states needs to be *nested* using the attention mask. Once they are nested, the attention mask is not needed anymore, therefore can be set to `None`. This is how the forward pass is built for `Bert`, these lines should remain pretty much similar across models, but sometimes the shapes of the attention masks are different across models.
129129
```python
130130
super().forward_checker()
131131

0 commit comments

Comments
 (0)