Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gptq refactor #1770

Merged
merged 5 commits into from
May 7, 2024
Merged

Gptq refactor #1770

merged 5 commits into from
May 7, 2024

Conversation

xin3he
Copy link
Contributor

@xin3he xin3he commented Apr 30, 2024

Type of Change

feature
API changed or not: yes

Description

  • Reduce additional initialization of quantizer between prepare and convert
    Before:
    image
    Now:
    Reduce additional initialization of quantizer between prepare and convert
    image

  • Migrate GPTQ to Torch new 3x API

GPTQ

from neural_compressor.torch.quantization import get_default_gptq_config, prepare, convert
quant_config = get_default_gptq_config()
model= prepare(model, quant_config)
run_fn(model)
q_model = convert(model)

How has this PR been tested?

Pre-ci

@xin3he xin3he requested review from Kaihui-intel, yuwenzho and yiliu30 and removed request for yuwenzho April 30, 2024 12:01
Copy link

github-actions bot commented Apr 30, 2024

⚡ Required checks status: All passing 🟢

Groups summary

🟢 Code Scan Tests workflow
Check ID Status Error details
Code-Scan success
Code-Scan (Bandit Code Scan Bandit) success
Code-Scan (DocStyle Code Scan DocStyle) success
Code-Scan (Pylint Code Scan Pylint) success

These checks are required after the changes to neural_compressor/torch/algorithms/weight_only/gptq.py, neural_compressor/torch/quantization/algorithm_entry.py.

🟢 Model Tests 3x workflow
Check ID Status Error details
Model-Test-3x success
Model-Test-3x (Generate Report GenerateReport) success
Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4) success
Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4_dq_bnb) success
Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4_dq_ggml) success

These checks are required after the changes to neural_compressor/torch/algorithms/weight_only/gptq.py, neural_compressor/torch/quantization/algorithm_entry.py.

🟢 Unit Tests 3x-PyTorch workflow
Check ID Status Error details
UT-3x-Torch success
UT-3x-Torch (Coverage Compare CollectDatafiles) success
UT-3x-Torch (Unit Test 3x Torch Unit Test 3x Torch) success
UT-3x-Torch (Unit Test 3x Torch baseline Unit Test 3x Torch baseline) success

These checks are required after the changes to neural_compressor/torch/algorithms/weight_only/gptq.py, neural_compressor/torch/quantization/algorithm_entry.py, test/3x/torch/quantization/weight_only/test_gptq.py.


Thank you for your contribution! 💜

Note
This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact chensuyue or XuehaoSun for help.

@xin3he
Copy link
Contributor Author

xin3he commented May 6, 2024

Final decision:
image

@yiliu30
Copy link
Contributor

yiliu30 commented May 6, 2024

One more thing, suggest merging it into master directly.

xin3he added 3 commits May 6, 2024 16:37
Signed-off-by: xin3he <xin3.he@intel.com>
Signed-off-by: xin3he <xin3.he@intel.com>
Signed-off-by: xin3he <xin3.he@intel.com>
@xin3he xin3he changed the base branch from yuwenzho/refactor_rtn_hqq_awq to master May 6, 2024 09:34
xin3he added 2 commits May 6, 2024 17:39
Signed-off-by: xin3he <xin3.he@intel.com>
Signed-off-by: xin3he <xin3.he@intel.com>
@xin3he xin3he merged commit 84d7055 into master May 7, 2024
30 checks passed
@xin3he xin3he deleted the gptq_refactor branch May 7, 2024 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants