[MinMax] Embedding nodes as input nodes for inference graph #3320
+37,308
−28,795
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
NNCF PT
daniil-lyakhov
force-pushed
the
dl/shape_of_sub_emb_fix
branch
from
9689dca to
0a2f240
Compare
AlexanderDokuchaev
requested changes
Mar 5, 2025
| @@ -50,6 +51,7 @@ | |||
| from nncf.torch.quantization.layers import BaseQuantizer | |||
| from nncf.torch.quantization.layers import PTQuantizerSpec | |||
| from nncf.torch.quantization.layers import get_scale_shape | |||
| from nncf.torch.utils import get_weight_nodes_in_inference_grpah | |||
There was a problem hiding this comment.
Suggested change
| from nncf.torch.utils import get_weight_nodes_in_inference_grpah | |
| from nncf.torch.utils import get_weight_nodes_in_inference_graph |
nncf/torch/utils.py
Outdated
| @@ -467,3 +470,46 @@ def get_model_dtype(model: torch.nn.Module) -> torch.dtype: | |||
| # The model had no parameters at all, assume FP32 | |||
| dtype = torch.float32 | |||
| return dtype | |||
|
|
|||
|
|
|||
| def get_weight_nodes_in_inference_grpah( | |||
There was a problem hiding this comment.
Suggested change
| def get_weight_nodes_in_inference_grpah( | |
| def get_weight_nodes_in_inference_graph( |
| @@ -86,7 +86,6 @@ def quantize_impl( | |||
| advanced_parameters=advanced_parameters, | |||
| ) | |||
|
|
|||
| # To make it easier for bias correction algorithms. | |||
There was a problem hiding this comment.
Why was this comment removed?
nncf/torch/utils.py
Outdated
|
|
||
| # Inference graph does not containt constans, so | ||
| # any missed input edge means it is a constant branch. | ||
| return node.metatype in [om.PTMatMulMetatype, om.PTAddmmMetatype] and len( |
There was a problem hiding this comment.
Please use variables to make it more readable
nncf/torch/utils.py
Outdated
| # any missed input edge means it is a constant branch. | ||
| return node.metatype in [om.PTMatMulMetatype, om.PTAddmmMetatype] and len( | ||
| inference_nncf_graph.get_input_edges(node) | ||
| ) < len(node.metatype.weight_port_ids) |
There was a problem hiding this comment.
is it work for possible missed inputs, that determinate in get_nodes_with_missed_input_edges?
nncf/torch/utils.py
Outdated
|
|
||
|
|
||
| def get_weight_nodes_in_inference_grpah( | ||
| inference_nncf_graph: NNCFGraph, mat_mul_metatypes: List[om.PTOperatorMetatype] |
There was a problem hiding this comment.
Looks like mat_mul_metatypes expected always same value, mat_mul_metatypes should not be used as argument instead use reusable constant variable
Co-authored-by: Alexander Dokuchaev <alexander.dokuchaev@intel.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reopen of the #2862
Changes
get_weight_nodesmethod is updated to work with the inference graphOpenVINOQuantizerand FXnncf.quantizeimplementationReason for changes
Related tickets
163025
Tests