Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate _get_compression_options #563

Merged
merged 1 commit into from
Feb 16, 2024
Merged

Deprecate _get_compression_options #563

merged 1 commit into from
Feb 16, 2024

Conversation

echarlaix
Copy link
Collaborator

@echarlaix echarlaix commented Feb 15, 2024

In this PR :

  • we enable the ov_config argument for all OVModels (previously reserved to OVModelForCausalLM) in order to give the possibility to pass any quantization_config
  • we deprecate the _get_compression_options method

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@echarlaix echarlaix marked this pull request as ready for review February 15, 2024 18:11
@echarlaix echarlaix requested a review from AlexKoff88 February 15, 2024 18:17
@AlexKoff88
Copy link
Collaborator

@echarlaix, what is the plan for CLI? Will we have all the granular options exposed to the user, e.g. --bits, --sym, --dataset, etc.

@echarlaix
Copy link
Collaborator Author

@echarlaix, what is the plan for CLI? Will we have all the granular options exposed to the user, e.g. --bits, --sym, --dataset, etc.

Yes I'm working on a PR to deprecate compression_options and compression_ratio, will then do a second PR to deprecate add --bits, --sym parameters

@echarlaix echarlaix merged commit 3daccec into main Feb 16, 2024
10 of 12 checks passed
@echarlaix echarlaix deleted the deprecate-load-8-bit branch February 16, 2024 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants