Skip to content

Commit e470f6c

Browse files
xin3hexin3he
authored
[3x] add recommendation examples (#1844)
Signed-off-by: xin3he <xin3.he@intel.com>
1 parent a141512 commit e470f6c

File tree

19 files changed

+4220
-63
lines changed

19 files changed

+4220
-63
lines changed

‎docs/3x/PT_MXQuant.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ user_model = convert(model=user_model)
9595

9696
## Examples
9797

98-
- PyTorch [huggingface models](/examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx)
98+
- PyTorch [huggingface models](/examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx_quant)
9999

100100

101101
## Reference

‎examples/.config/model_params_pytorch_3x.json

+53-46
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,53 @@
1-
{
2-
"pytorch": {
3-
"gpt_j_ipex":{
4-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
5-
"dataset_location": "",
6-
"input_model": "",
7-
"main_script": "run_clm_no_trainer.py",
8-
"batch_size": 1
9-
},
10-
"gpt_j_ipex_sq":{
11-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
12-
"dataset_location": "",
13-
"input_model": "",
14-
"main_script": "run_clm_no_trainer.py",
15-
"batch_size": 1
16-
},
17-
"llama2_7b_ipex":{
18-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
19-
"dataset_location": "",
20-
"input_model": "",
21-
"main_script": "run_clm_no_trainer.py",
22-
"batch_size": 1
23-
},
24-
"llama2_7b_ipex_sq":{
25-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
26-
"dataset_location": "",
27-
"input_model": "",
28-
"main_script": "run_clm_no_trainer.py",
29-
"batch_size": 1
30-
},
31-
"opt_125m_ipex":{
32-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
33-
"dataset_location": "",
34-
"input_model": "",
35-
"main_script": "run_clm_no_trainer.py",
36-
"batch_size": 8
37-
},
38-
"opt_125m_ipex_sq":{
39-
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
40-
"dataset_location": "",
41-
"input_model": "",
42-
"main_script": "run_clm_no_trainer.py",
43-
"batch_size": 8
44-
}
45-
}
46-
}
1+
{
2+
"pytorch": {
3+
"gpt_j_ipex":{
4+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
5+
"dataset_location": "",
6+
"input_model": "",
7+
"main_script": "run_clm_no_trainer.py",
8+
"batch_size": 1
9+
},
10+
"gpt_j_ipex_sq":{
11+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
12+
"dataset_location": "",
13+
"input_model": "",
14+
"main_script": "run_clm_no_trainer.py",
15+
"batch_size": 1
16+
},
17+
"llama2_7b_ipex":{
18+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
19+
"dataset_location": "",
20+
"input_model": "",
21+
"main_script": "run_clm_no_trainer.py",
22+
"batch_size": 1
23+
},
24+
"llama2_7b_ipex_sq":{
25+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
26+
"dataset_location": "",
27+
"input_model": "",
28+
"main_script": "run_clm_no_trainer.py",
29+
"batch_size": 1
30+
},
31+
"opt_125m_ipex":{
32+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/static_quant",
33+
"dataset_location": "",
34+
"input_model": "",
35+
"main_script": "run_clm_no_trainer.py",
36+
"batch_size": 8
37+
},
38+
"opt_125m_ipex_sq":{
39+
"model_src_dir": "nlp/huggingface_models/language-modeling/quantization/smooth_quant",
40+
"dataset_location": "",
41+
"input_model": "",
42+
"main_script": "run_clm_no_trainer.py",
43+
"batch_size": 8
44+
},
45+
"dlrm_ipex": {
46+
"model_src_dir": "recommendation/dlrm/static_quant/ipex",
47+
"dataset_location": "/mnt/local_disk3/dataset/dlrm/dlrm/input",
48+
"input_model": "/mnt/local_disk3/dataset/dlrm/dlrm/dlrm_weight/tb00_40M.pt",
49+
"main_script": "dlrm_s_pytorch.py",
50+
"batch_size": 16384
51+
}
52+
}
53+
}

‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx/README.md ‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx_quant/README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Run
22

33
## Run WOQ MX FP4 model
4+
45
``` python
56
python run_clm_no_trainer.py --model [model_name_or_id] --quantize --accuracy --tasks lambada_openai --w_dtype fp4 --woq
6-
```
7+
```

‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx/requirements.txt ‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx_quant/requirements.txt

File renamed without changes.

‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx/run_clm_no_trainer.py ‎examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx_quant/run_clm_no_trainer.py

File renamed without changes.

‎examples/3.x_api/pytorch/recommendation/dlrm/static_quant/ipex/CODE_OF_CONDUCT.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Code of Conduct
2+
3+
Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
4+
Please read the [full text](https://code.fb.com/codeofconduct/)
5+
so that you can understand what actions will and will not be tolerated.

‎examples/3.x_api/pytorch/recommendation/dlrm/static_quant/ipex/CONTRIBUTING.md

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Contributing to DLRM
2+
We want to make contributing to this project as easy and transparent as
3+
possible.
4+
5+
## Pull Requests
6+
We actively welcome your pull requests.
7+
8+
1. Fork the repo and create your branch from `master`.
9+
2. If you've added code that should be tested, add tests.
10+
3. If you've changed APIs, update the documentation.
11+
4. Ensure the test suite passes.
12+
5. Make sure your code lints.
13+
6. If you haven't already, complete the Contributor License Agreement ("CLA").
14+
15+
## Contributor License Agreement ("CLA")
16+
In order to accept your pull request, we need you to submit a CLA. You only need
17+
to do this once to work on any of Facebook's open source projects.
18+
19+
Complete your CLA here: <https://code.facebook.com/cla>
20+
21+
## Issues
22+
We use GitHub issues to track public bugs. Please ensure your description is
23+
clear and has sufficient instructions to be able to reproduce the issue.
24+
25+
Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
26+
disclosure of security bugs. In those cases, please go through the process
27+
outlined on that page and do not file a public issue.
28+
29+
## Coding Style
30+
* 4 spaces for indentation rather than tabs
31+
* 80 character line length
32+
* in general, please maintain a consistent style with the rest of the code
33+
34+
## License
35+
By contributing to DLRM, you agree that your contributions will be licensed
36+
under the LICENSE file in the root directory of this source tree.

‎examples/3.x_api/pytorch/recommendation/dlrm/static_quant/ipex/LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) Facebook, Inc. and its affiliates.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎examples/3.x_api/pytorch/recommendation/dlrm/static_quant/ipex/README.md

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
Step-by-Step
2+
============
3+
4+
This document is used to list steps of reproducing PyTorch DLRM tuning zoo result. and original DLRM README is in [DLRM README](https://github.com/facebookresearch/dlrm/blob/master/README.md)
5+
6+
> **Note**
7+
>
8+
> Please ensure your PC have >370G memory to run DLRM
9+
> IPEX version >= 1.11
10+
11+
# Prerequisite
12+
13+
### 1. Environment
14+
15+
PyTorch 1.11 or higher version is needed with pytorch_fx backend.
16+
17+
```shell
18+
# Install dependency
19+
cd examples/pytorch/recommendation/dlrm/quantization/ptq/ipex
20+
pip install -r requirements.txt
21+
```
22+
> Note: Validated PyTorch [Version](/docs/source/installation_guide.md#validated-software-environment).
23+
24+
### 2. Prepare Dataset
25+
26+
The code supports interface with the [Criteo Terabyte Dataset](https://labs.criteo.com/2013/12/download-terabyte-click-logs/)
27+
28+
1. download the raw data files day_0.gz, ...,day_23.gz and unzip them.
29+
2. Specify the location of the unzipped text files day_0, ...,day_23, using --raw-data-file=<path/day> (the day number will be appended automatically), please refer "Run" command.
30+
31+
### 3. Prepare pretrained model
32+
33+
Download the DLRM PyTorch weights (`tb00_40M.pt`, 90GB) from the
34+
[MLPerf repo](https://github.com/mlcommons/inference/tree/master/recommendation/dlrm/pytorch#more-information-about-the-model-weights)
35+
36+
# Run
37+
### tune with INC
38+
```shell
39+
cd examples/pytorch/recommendation/dlrm/quantization/ptq/ipex
40+
bash run_quant.sh --input_model="/path/of/pretrained/model" --dataset_location="/path/of/dataset"
41+
```
42+
43+
### benchmark
44+
```shell
45+
bash run_benchmark.sh --input_model="/path/of/pretrained/model" --dataset_location="/path/of/dataset" --mode=accuracy --int8=true
46+
```
47+
48+
49+
Examples of enabling Intel® Neural Compressor
50+
=========================
51+
52+
This is a tutorial of how to enable DLRM model with Intel® Neural Compressor.
53+
54+
55+
### Code update
56+
57+
We need update dlrm_s_pytorch.py like below
58+
59+
```python
60+
# evaluation
61+
def eval_func(model):
62+
args.int8 = model.is_quantized
63+
with torch.no_grad():
64+
return inference(
65+
args,
66+
model,
67+
best_acc_test,
68+
best_auc_test,
69+
test_ld,
70+
trace=args.int8
71+
)
72+
73+
# calibration
74+
def calib_fn(model):
75+
calib_number = 0
76+
for X_test, lS_o_test, lS_i_test, T in train_ld:
77+
if calib_number < 102400:
78+
model(X_test, lS_o_test, lS_i_test)
79+
calib_number += 1
80+
81+
from neural_compressor.torch.quantization import SmoothQuantConfig, autotune, TuningConfig
82+
tune_config = TuningConfig(config_set=SmoothQuantConfig.get_config_set_for_tuning())
83+
dlrm = autotune(
84+
dlrm,
85+
tune_config=tune_config,
86+
eval_fn=eval_func,
87+
run_fn=calib_fn,
88+
)
89+
dlrm.save("saved_results")
90+
```

0 commit comments

Comments
 (0)