Skip to content

Commit 498af74

Browse files
XuehaoSunchensuyue
XuehaoSun
and
chensuyue
authored
Update SQ/WOQ status (#1869)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com>
1 parent b401b02 commit 498af74

File tree

1 file changed

+132
-134
lines changed

1 file changed

+132
-134
lines changed

‎docs/source/llm_recipes.md

+132-134
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ This document aims to publish the specific recipes we achieved for the popular L
2929
| databricks/dolly-v2-12b | ✖ | ✔ | ✖ |
3030
| EleutherAI/gpt-neox-20b | ✖ | ✔ | ✔ |
3131
| mistralai/Mistral-7B-v0.1 | ✖ | ✔ | ✔ |
32-
| THUDM/chatglm2-6b | WIP | ✔ | ✔ |
33-
| THUDM/chatglm3-6b | WIP | ✔ | ✔ |
32+
| THUDM/chatglm2-6b | ✔ | ✔ | ✔ |
33+
| THUDM/chatglm3-6b | WIP | ✔ | WIP |
3434

3535
**Detail recipes can be found [HERE](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/text-generation/quantization/llm_quantization_recipes.md).**
3636

@@ -40,8 +40,8 @@ This document aims to publish the specific recipes we achieved for the popular L
4040
> - The WIP recipes will be published soon.
4141
4242
## Large Language Models Accuracy
43-
<table>
44-
<thead>
43+
44+
<table><thead>
4545
<tr>
4646
<th rowspan="3">Model</th>
4747
<th colspan="9">lambada_openai</th>
@@ -63,212 +63,210 @@ This document aims to publish the specific recipes we achieved for the popular L
6363
<th>Ratio</th>
6464
<th>ACC</th>
6565
<th>Ratio</th>
66-
</tr>
67-
</thead>
66+
</tr></thead>
6867
<tbody>
6968
<tr>
7069
<td>baichuan-inc/Baichuan-13B-Chat</td>
7170
<td>67.57%</td>
72-
<td>68.23%</td>
73-
<td>1.0098</td>
74-
<td>67.57%</td>
75-
<td>1.0000</td>
76-
<td>67.84%</td>
77-
<td>1.0040</td>
78-
<td>NA</td>
79-
<td>NA</td>
71+
<td>69.07%</td>
72+
<td>1.0222</td>
73+
<td>67.55%</td>
74+
<td>0.9997</td>
75+
<td>68.12%</td>
76+
<td>1.0081</td>
77+
<td>66.93%</td>
78+
<td>0.9905</td>
8079
</tr>
8180
<tr>
8281
<td>baichuan-inc/Baichuan2-13B-Chat</td>
8382
<td>71.51%</td>
84-
<td>70.89%</td>
85-
<td>0.9913</td>
86-
<td>71.53%</td>
87-
<td>1.0003</td>
88-
<td>71.76%</td>
89-
<td>1.0035</td>
90-
<td>NA</td>
91-
<td>NA</td>
83+
<td>75.57%</td>
84+
<td>1.0568</td>
85+
<td>71.57%</td>
86+
<td>1.0008</td>
87+
<td>70.81%</td>
88+
<td>0.9902</td>
89+
<td>N/A</td>
90+
<td>N/A</td>
9291
</tr>
9392
<tr>
9493
<td>baichuan-inc/Baichuan2-7B-Chat</td>
9594
<td>67.67%</td>
96-
<td>67.96%</td>
97-
<td>1.0043</td>
98-
<td>67.59%</td>
99-
<td>0.9988</td>
100-
<td>67.24%</td>
101-
<td>0.9936</td>
102-
<td>67.42%</td>
103-
<td>0.9963</td>
95+
<td>68.06%</td>
96+
<td>1.0058</td>
97+
<td>67.61%</td>
98+
<td>0.9991</td>
99+
<td>67.90%</td>
100+
<td>1.0034</td>
101+
<td>N/A</td>
102+
<td>N/A</td>
104103
</tr>
105104
<tr>
106105
<td>bigscience/bloom-1b7</td>
107106
<td>46.34%</td>
108107
<td>47.99%</td>
109108
<td>1.0356</td>
110-
<td>46.38%</td>
111-
<td>1.0009</td>
112-
<td>46.19%</td>
113-
<td>0.9968</td>
114-
<td>NA</td>
115-
<td>NA</td>
109+
<td>46.21%</td>
110+
<td>0.9972</td>
111+
<td>46.90%</td>
112+
<td>1.0121</td>
113+
<td>N/A</td>
114+
<td>N/A</td>
116115
</tr>
117116
<tr>
118117
<td>databricks/dolly-v2-12b</td>
119118
<td>64.35%</td>
120-
<td>NA</td>
121-
<td>NA</td>
122-
<td>64.10%</td>
123-
<td>0.9961</td>
124-
<td>NA</td>
125-
<td>NA</td>
126-
<td>NA</td>
127-
<td>NA</td>
119+
<td>N/A</td>
120+
<td>N/A</td>
121+
<td>63.92%</td>
122+
<td>0.9933</td>
123+
<td>N/A</td>
124+
<td>N/A</td>
125+
<td>N/A</td>
126+
<td>N/A</td>
128127
</tr>
129128
<tr>
130129
<td>EleutherAI/gpt-j-6b</td>
131130
<td>68.31%</td>
132-
<td>68.33%</td>
133-
<td>1.0003</td>
134-
<td>68.23%</td>
135-
<td>0.9988</td>
136-
<td>68.79%</td>
137-
<td>1.0070</td>
138-
<td>68.43%</td>
139-
<td>1.0018</td>
131+
<td>68.27%</td>
132+
<td>0.9994</td>
133+
<td>68.27%</td>
134+
<td>0.9994</td>
135+
<td>68.35%</td>
136+
<td>1.0006</td>
137+
<td>68.02%</td>
138+
<td>0.9958</td>
140139
</tr>
141140
<tr>
142141
<td>EleutherAI/gpt-neox-20b</td>
143142
<td>72.33%</td>
144-
<td>NA</td>
145-
<td>NA</td>
146-
<td>72.25%</td>
147-
<td>0.9989</td>
148-
<td>71.96%</td>
149-
<td>0.9949</td>
150-
<td>NA</td>
151-
<td>NA</td>
143+
<td>N/A</td>
144+
<td>N/A</td>
145+
<td>72.29%</td>
146+
<td>0.9994</td>
147+
<td>71.74%</td>
148+
<td>0.9918</td>
149+
<td>N/A</td>
150+
<td>N/A</td>
152151
</tr>
153152
<tr>
154153
<td>facebook/opt-1.3b</td>
155154
<td>57.89%</td>
156-
<td>57.54%</td>
157-
<td>0.9940</td>
158-
<td>58.08%</td>
159-
<td>1.0033</td>
160-
<td>58.57%</td>
161-
<td>1.0117</td>
162-
<td>NA</td>
163-
<td>NA</td>
155+
<td>57.68%</td>
156+
<td>0.9964</td>
157+
<td>58.12%</td>
158+
<td>1.0040</td>
159+
<td>58.26%</td>
160+
<td>1.0064</td>
161+
<td>N/A</td>
162+
<td>N/A</td>
164163
</tr>
165164
<tr>
166165
<td>facebook/opt-30b</td>
167166
<td>71.49%</td>
168-
<td>71.51%</td>
169-
<td>1.0003</td>
170-
<td>71.51%</td>
171-
<td>1.0003</td>
172-
<td>71.82%</td>
173-
<td>1.0046</td>
174-
<td>72.11%</td>
175-
<td>1.0087</td>
167+
<td>71.78%</td>
168+
<td>1.0041</td>
169+
<td>71.53%</td>
170+
<td>1.0006</td>
171+
<td>71.59%</td>
172+
<td>1.0014</td>
173+
<td>71.80%</td>
174+
<td>1.0043</td>
176175
</tr>
177176
<tr>
178177
<td>meta-llama/Llama-2-13b-hf</td>
179178
<td>76.77%</td>
180179
<td>76.25%</td>
181180
<td>0.9932</td>
182-
<td>76.75%</td>
183-
<td>0.9997</td>
184-
<td>77.43%</td>
185-
<td>1.0086</td>
186-
<td>76.75%</td>
187-
<td>0.9997</td>
181+
<td>76.89%</td>
182+
<td>1.0016</td>
183+
<td>77.66%</td>
184+
<td>1.0116</td>
185+
<td>76.60%</td>
186+
<td>0.9978</td>
188187
</tr>
189188
<tr>
190189
<td>meta-llama/Llama-2-70b-hf</td>
191190
<td>79.64%</td>
192-
<td>79.55%</td>
193-
<td>0.9989</td>
194-
<td>79.57%</td>
195-
<td>0.9991</td>
191+
<td>79.14%</td>
192+
<td>0.9937</td>
193+
<td>79.62%</td>
194+
<td>0.9997</td>
196195
<td>80.09%</td>
197196
<td>1.0057</td>
198-
<td>79.97%</td>
199-
<td>1.0041</td>
197+
<td>79.68%</td>
198+
<td>1.0005</td>
200199
</tr>
201200
<tr>
202201
<td>meta-llama/Llama-2-7b-hf</td>
203202
<td>73.92%</td>
204203
<td>73.45%</td>
205204
<td>0.9936</td>
206-
<td>73.96%</td>
207-
<td>1.0005</td>
208-
<td>73.45%</td>
209-
<td>0.9936</td>
210-
<td>73.49%</td>
211-
<td>0.9942</td>
205+
<td>73.90%</td>
206+
<td>0.9997</td>
207+
<td>73.84%</td>
208+
<td>0.9989</td>
209+
<td>N/A</td>
210+
<td>N/A</td>
212211
</tr>
213212
<tr>
214213
<td>mistralai/Mistral-7B-v0.1</td>
215214
<td>75.90%</td>
216-
<td>NA</td>
217-
<td>NA</td>
215+
<td>N/A</td>
216+
<td>N/A</td>
218217
<td>75.80%</td>
219218
<td>0.9987</td>
220-
<td>76.13%</td>
221-
<td>1.0030</td>
222-
<td>75.61%</td>
223-
<td>0.9962</td>
219+
<td>76.25%</td>
220+
<td>1.0046</td>
221+
<td>75.74%</td>
222+
<td>0.9979</td>
224223
</tr>
225224
<tr>
226225
<td>THUDM/chatglm2-6b</td>
227226
<td>53.23%</td>
228-
<td>NA</td>
229-
<td>NA</td>
230-
<td>53.19%</td>
231-
<td>0.9992</td>
232-
<td>52.77%</td>
233-
<td>0.9914</td>
234-
<td>53.35%</td>
235-
<td>1.0023</td>
227+
<td>52.86%</td>
228+
<td>0.9930</td>
229+
<td>53.00%</td>
230+
<td>0.9957</td>
231+
<td>52.90%</td>
232+
<td>0.9938</td>
233+
<td>52.92%</td>
234+
<td>0.9942</td>
236235
</tr>
237236
<tr>
238237
<td>THUDM/chatglm3-6b</td>
239238
<td>59.09%</td>
240-
<td>NA</td>
241-
<td>NA</td>
242-
<td>59.01%</td>
243-
<td>0.9986</td>
244-
<td>NA</td>
245-
<td>NA</td>
246-
<td>58.61%</td>
247-
<td>0.9919</td>
239+
<td>N/A</td>
240+
<td>N/A</td>
241+
<td>59.03%</td>
242+
<td>0.9990</td>
243+
<td>N/A</td>
244+
<td>N/A</td>
245+
<td>N/A</td>
246+
<td>N/A</td>
248247
</tr>
249248
<tr>
250249
<td>tiiuae/falcon-40b</td>
251250
<td>77.22%</td>
252-
<td>77.04%</td>
253-
<td>0.9977</td>
254-
<td>77.22%</td>
255-
<td>1.0000</td>
256-
<td>77.94%</td>
257-
<td>1.0093</td>
258-
<td>78.79%</td>
259-
<td>1.0203</td>
251+
<td>76.95%</td>
252+
<td>0.9965</td>
253+
<td>77.18%</td>
254+
<td>0.9995</td>
255+
<td>77.55%</td>
256+
<td>1.0043</td>
257+
<td>77.82%</td>
258+
<td>1.0078</td>
260259
</tr>
261260
<tr>
262261
<td>tiiuae/falcon-7b</td>
263262
<td>74.67%</td>
264-
<td>76.44%</td>
265-
<td>1.0237</td>
266-
<td>74.77%</td>
267-
<td>1.0013</td>
268-
<td>75.00%</td>
269-
<td>1.0044</td>
270-
<td>NA</td>
271-
<td>NA</td>
263+
<td>76.63%</td>
264+
<td>1.0262</td>
265+
<td>74.73%</td>
266+
<td>1.0008</td>
267+
<td>75.06%</td>
268+
<td>1.0052</td>
269+
<td>74.00%</td>
270+
<td>0.9910</td>
272271
</tr>
273-
</tbody>
274-
</table>
272+
</tbody></table>

0 commit comments

Comments
 (0)