Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Good First Issue][NNCF][Tests]: Refactor weights compression tests into a template tests. #3129

Open
daniil-lyakhov opened this issue Dec 3, 2024 · 21 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@daniil-lyakhov
Copy link
Collaborator

Context

Majority of the NNCF PTQ tests are using templates: for example tests for Fast Bias Correction or MinMax algorithm.
The structure is following: there is a template file like this, and each backend implements abstract methods of the template file like this.

What needs to be done?

The task is to:

  1. Introduce Template tests(class TemplateTestWeightsCompression) for weights compression in the test_weights_compression.py file in this directory: https://github.com/openvinotoolkit/nncf/tree/develop/tests/cross_fw/test_templates
  2. Refactor this files
    ** https://github.com/openvinotoolkit/nncf/blob/develop/tests/torch/ptq/test_weights_compression.py
    ** https://github.com/openvinotoolkit/nncf/blob/develop/tests/torch/fx/test_compress_weights.py
    to implement the template.
    The template should implement following common tests:
  • test_compress_weights
  • test_compress_shared_weights / test_compress_weights_shared_weights
  • test_compress_weights_functional_model / test_compress_weights_functional_model
  • test_raise_error_with_unsupported_params_for_int8
  • test_raise_error_with_unsupported_params_for_int4
  • test_raise_error_with_not_int8
  • test_raise_error_for_statistics_caching
  • test_get_dtype_attribute_of_parameter
  • test_model_devices_and_precisions

Please use this two template files as a reference:
https://github.com/openvinotoolkit/nncf/blob/develop/tests/torch/fx/test_fast_bias_correction.py
https://github.com/openvinotoolkit/nncf/blob/develop/tests/torch/ptq/test_fast_bias_correction.py

Example Pull Requests

#2856

Resources

Contact points

@daniil-lyakhov

Ticket

No response

@daniil-lyakhov daniil-lyakhov added the good first issue Good for newcomers label Dec 3, 2024
@daniil-lyakhov
Copy link
Collaborator Author

CC: AHB102

@github-project-automation github-project-automation bot moved this to Contributors Needed in Good first issues Dec 4, 2024
@rk119
Copy link
Contributor

rk119 commented Dec 7, 2024

Hi @AHB102,

Are you working on this issue? :)

@AHB102
Copy link

AHB102 commented Dec 7, 2024

@rk119 Yeah.

@rk119
Copy link
Contributor

rk119 commented Dec 7, 2024

@rk119 Yeah.

Alright!

@AHB102
Copy link

AHB102 commented Dec 7, 2024

.take

Copy link

github-actions bot commented Dec 7, 2024

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

@daniil-lyakhov daniil-lyakhov moved this from Contributors Needed to Assigned in Good first issues Dec 12, 2024
@rk119
Copy link
Contributor

rk119 commented Jan 17, 2025

Hi @AHB102,

Are you still working on this issue?

@AHB102
Copy link

AHB102 commented Jan 17, 2025

@rk119 I've started on this, but I'm tied up with other things at the moment. Could you please take care of this? Sorry for the inconvenience.

@rk119
Copy link
Contributor

rk119 commented Jan 17, 2025

@AHB102 it's alright. I presume that this issue is complete in another PR already :)

@ljaljushkin
Copy link
Contributor

@AHB102 it's alright. I presume that this issue is complete in another PR already :)

@AHB102 @rk119 #3179 doesn't resolve the issue completely. There are still duplicated tests to make template ones.

@rk119
Copy link
Contributor

rk119 commented Feb 2, 2025

@ljaljushkin Okay then, I would like to work on this issue as well :) if it's alright

@ljaljushkin
Copy link
Contributor

@rk119 I've started on this, but I'm tied up with other things at the moment. Could you please take care of this? Sorry for the inconvenience.

@AHB102 can I reassign it to @rk119 then?

@shumaari
Copy link
Contributor

shumaari commented Feb 3, 2025

@ljaljushkin @rk119 @AHB102 May I help out on this issue?

@ljaljushkin
Copy link
Contributor

ljaljushkin commented Feb 4, 2025

@ljaljushkin @rk119 @AHB102 May I help out on this issue?

@shumaari I'm afraid not, issue is quite straightforward for splitting.
Feel free to take brand-new GFI #3249

@shumaari
Copy link
Contributor

shumaari commented Feb 4, 2025

@ljaljushkin thank you, will take #3249 instead.

@p-wysocki
Copy link
Contributor

p-wysocki commented Mar 5, 2025

Let's unassign that task due to current assignee's inactivity.

@rk119
Copy link
Contributor

rk119 commented Mar 20, 2025

@ljaljushkin Feel free to leave this issue open to other potential contributors as I have other commitments to attend to presently :)

@ljaljushkin ljaljushkin moved this from Assigned to Contributors Needed in Good first issues Mar 20, 2025
@raghavx1
Copy link

.take

Copy link

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

@alexsu52 alexsu52 moved this from Contributors Needed to Assigned in Good first issues Mar 20, 2025
@raghavx1
Copy link

Hi @ljaljushkin @alexsu52
I have started working on the problem and have prepared a starting version of the template can you please have a look and check if I'm going in the right direction.

Link to changes: develop...raghavx1:nncf:develop

Thank You
Raghav Sharma

@ljaljushkin
Copy link
Contributor

ljaljushkin commented Mar 24, 2025

Hi @ljaljushkin @alexsu52 I have started working on the problem and have prepared a starting version of the template can you please have a look and check if I'm going in the right direction.

Link to changes: develop...raghavx1:nncf:develop

Thank You Raghav Sharma

+ @daniil-lyakhov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
Status: Assigned
Development

No branches or pull requests

7 participants