@@ -22,7 +22,7 @@ It is a `set of compression algorithms <https://github.com/openvinotoolkit/nncf/
22
22
organized as a Python package, that make your models smaller and faster. Note that NNCF
23
23
is **not part of the OpenVINO package **, so it needs to be installed separately. It supports
24
24
models in **PyTorch **, **TensorFlow ** , **ONNX **, and **OpenVINO IR ** formats, offering
25
- the following optimizations:
25
+ the following main optimizations:
26
26
27
27
.. image :: ../assets/images/WHAT_TO_USE.svg
28
28
@@ -42,20 +42,41 @@ the following optimizations:
42
42
as Quantization-aware Training. This kind of optimization requires the use of the model's
43
43
original framework, for NNCF, it is either PyTorch or TensorFlow.
44
44
45
- A common approach is to perform post-training quantization first, as it is the easiest option.
46
- If the result proves unsatisfactory, quantization-aware training will give you higher accuracy
47
- with the same level of performance boost. For the most performant product, adding filter pruning
48
- will further streamline the model.
49
45
50
- To learn about the full scope of the framework, its installation, and technical details, visit
51
- both `the NNCF repository <https://github.com/openvinotoolkit/nncf?tab=readme-ov-file >`__ and
52
- `NNCF API documentation <https://openvinotoolkit.github.io/nncf/autoapi/nncf/ >`__.
46
+
47
+ Recommended workflows
48
+ ##########################
49
+
50
+ * A common approach for most cases is to:
51
+
52
+ 1. Perform post-training quantization first, as it is the easiest option.
53
+ 2. For even better results, combine post-training quantization with filter pruning.
54
+ 3. If the accuracy drop is unacceptable, use quantization-aware training instead. It will give
55
+ you the same level of performance boost, with a smaller impact on accuracy.
56
+
57
+ * **Weight compression ** works **only with LLMs **. Do not try to use it with other models.
58
+ * For **visual-multimodal ** use cases, the encoder / decoder split approach may be recommended.
59
+
60
+
61
+
62
+
63
+
53
64
54
65
55
66
56
67
.. image :: ../assets/images/DEVELOPMENT_FLOW_V3_crunch.svg
57
68
58
69
70
+
71
+ Installation and usage
72
+ ###########################
73
+
74
+ To learn about the full scope of the framework, its installation, and technical details, visit
75
+ both `the NNCF repository <https://github.com/openvinotoolkit/nncf?tab=readme-ov-file >`__ and
76
+ `NNCF API documentation <https://openvinotoolkit.github.io/nncf/autoapi/nncf/ >`__.
77
+
78
+
79
+
59
80
.. tab-set ::
60
81
61
82
.. tab-item :: Installation
0 commit comments