You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### Changes
Add [markdownlint](https://github.com/DavidAnson/markdownlint) check to
pre-commit
Fixed:
- Incorrect links
- Ordered lists
- Blank lines before and after headers and code blocks
- Types for code blocks
- Trailing spaces
- Spellcheck
- Removed module-timm_custom_modules from api doc
* Submitting issues against the current code to report bugs or request features
5
6
* Extending NNCF functionality with important features (e.g. to address community requests, improve usability, implement a recently published compression algorithm, etc.)
6
7
* Adding example scripts to showcase NNCF usage in real training pipelines and provide the means to reproduce the reported compression results
7
8
* Providing recipes (specific NNCF configurations and training hyperparameters) to obtain state-of-the-art compression using NNCF for existing models
8
9
* Adding well-defined patches that integrate NNCF into third-party repositories
9
-
* Reducing performance overhead of NNCF compression by writing specialized CUDA kernels for compression operations or improving existing ones.
10
+
* Reducing performance overhead of NNCF compression by writing specialized CUDA kernels for compression operations or improving existing ones.
10
11
11
12
The latter forms are accepted as pull requests from your own forks of the NNCF repository.
12
13
13
14
Any contributions must not violate the repository's [LICENSE](./LICENSE) requirements.
14
15
15
-
## Installation
16
-
### (Experimental) ONNXRuntime-OpenVINO
17
-
Install the package and its dependencies by running the following in the repository root directory:
18
-
```bash
19
-
make install-onnx-dev
20
-
```
21
-
22
16
## Testing
23
17
24
18
After your pull request is submitted, the maintainer will launch a scope of CI tests against it.
@@ -28,42 +22,30 @@ The pre-commit scope may be run locally by executing the `pytest` command (witho
28
22
Please run the pre-commit testing scope locally before submitting your PR and ensure that it passes to conserve your own time and that of the reviewing maintainer.
29
23
30
24
New feature pull requests should include all the necessary testing code.
31
-
Testing is done using the `pytest` framework.
25
+
Testing is done using the `pytest` framework.
32
26
The test files should be located inside the [tests](./tests) directory and start with `test_` so that the `pytest` is able to discover them.
33
27
Any additional data that is required for tests (configuration files, mock datasets, etc.) must be stored within the [tests/data](./tests/data) folder.
34
28
The test files themselves may be grouped in arbitrary directories according to their testing purpose and common sense.
35
29
36
-
Any additional tests in the [tests](./tests) directory will be automatically added into the pre-commit CI scope.
30
+
Any additional tests in the [tests](./tests) directory will be automatically added into the pre-commit CI scope.
37
31
If your testing code is more extensive than unit tests (in terms of test execution time), or would be more suited to be executed on a nightly/weekly basis instead of for each future commit, please inform the maintainers in your PR discussion thread so that our internal testing pipelines could be adjusted accordingly.
38
32
39
-
### Preset command for testing
40
-
You can launch appropriate tests against the framework by running the following command:
41
-
42
-
- (Experimental) ONNXRuntime-OpenVINO
43
-
```bash
44
-
test-onnx
45
-
```
46
-
47
33
## Code style
34
+
48
35
Changes to NNCF Python code should conform to [Python Style Guide](./docs/styleguide/PyGuide.md)
49
36
50
-
Pylint is used throughout the project to ensure code cleanliness and quality.
37
+
Pylint is used throughout the project to ensure code cleanliness and quality.
51
38
A Pylint run is also done as part of the pre-commit scope - the pre-commit `pytest` scope will not be run if your code fails the Pylint checks.
52
39
The Pylint rules and exceptions for this repository are described in the standard [.pylintrc](./.pylintrc) format - make sure your local linter uses these.
53
40
54
-
### Preset command for linting
55
-
You can launch appropriate linting against the framework by running the following command:
56
-
57
-
- (Experimental) ONNXRuntime-OpenVINO
58
-
```bash
59
-
pylint-onnx
60
-
```
61
-
62
41
## Binary files
63
-
Please refrain from adding huge binary files into the repository. If binary files have to be added, mark these to use Git LFS via the [.gitattributes](./.gitattributes) file.
42
+
43
+
Please refrain from adding huge binary files into the repository. If binary files have to be added, mark these to use Git LFS via the [.gitattributes](./.gitattributes) file.
64
44
65
45
## Model identifiers
46
+
66
47
When adding model configs and checkpoints to be showcased in NNCF's sample script, follow the format for naming these files:
48
+
67
49
1. The base name must be the same for the NNCF config file, AC config file, checkpoint file (PT/ONNX/OV) or checkpoint folder (TF), and other associated artifacts.
68
50
2. This name should be composed with the following format: `{model_name}_{dataset_name}` for FP32 models, `{topology_name}_{dataset_name}_{compression_algorithms_applied}`. The format may be extended if there are multiple models with the same topology, dataset and compression algos applied, which only differ in something else such as exact value of achieved sparsity. Align the naming of the new checkpoints with the existing ones.
69
-
3. Additional human-readable information on the model such as expected metrics and compression algorithm specifics (e.g. level of pruning/sparsity, per-tensor/per-channel quantizer configuration etc.) should be stored in a registry file (`tests/torch/sota_checkpoints_eval.json` for PT, `tests/tensorflow/sota_checkpoints_eval.json` for TF)
51
+
3. Additional human-readable information on the model such as expected metrics and compression algorithm specifics (e.g. level of pruning/sparsity, per-tensor/per-channel quantizer configuration etc.) should be stored in a registry file (`tests/torch/sota_checkpoints_eval.json` for PT, `tests/tensorflow/sota_checkpoints_eval.json` for TF)
@@ -21,13 +21,14 @@ Neural Network Compression Framework (NNCF) provides a suite of post-training an
21
21
22
22
NNCF is designed to work with models from [PyTorch](https://pytorch.org/), [TensorFlow](https://www.tensorflow.org/), [ONNX](https://onnx.ai/) and [OpenVINO™](https://docs.openvino.ai/latest/home.html).
23
23
24
-
NNCF provides [samples](#Model-compression-tutorials-and-samples) that demonstrate the usage of compression algorithms for different use cases and models. See compression results achievable with the NNCF-powered samples at [Model Zoo page](./docs/ModelZoo.md).
24
+
NNCF provides [samples](#model-compression-tutorials-and-samples) that demonstrate the usage of compression algorithms for different use cases and models. See compression results achievable with the NNCF-powered samples at [Model Zoo page](./docs/ModelZoo.md).
25
25
26
26
The framework is organized as a Python\* package that can be built and used in a standalone mode. The framework
27
27
architecture is unified to make it easy to add different compression algorithms for both PyTorch and TensorFlow deep
[//]: #(NNCF provides full [samples](#post-training-quantization-samples), which demonstrate Post-Training Quantization usage for PyTorch, TensorFlow, ONNX, OpenVINO.)
189
189
190
190
### Training-Time Compression
@@ -272,7 +272,8 @@ For a quicker start with NNCF-powered compression, try sample notebooks and scri
272
272
### Model Compression Tutorials
273
273
274
274
A collection of ready-to-run Jupyter* notebooks are available to demonstrate how to use NNCF compression algorithms to optimize models for inference with the OpenVINO Toolkit:
275
-
-[Accelerate Inference of NLP models with Post-Training Qunatization API of NNCF](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/105-language-quantize-bert)
275
+
276
+
-[Accelerate Inference of NLP models with Post-Training Quantization API of NNCF](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/105-language-quantize-bert)
276
277
-[Convert and Optimize YOLOv8 with OpenVINO](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/230-yolov8-optimization)
277
278
-[Convert and Optimize YOLOv7 with OpenVINO](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/226-yolov7-optimization)
278
279
-[NNCF Post-Training Optimization of Segment Anything Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/237-segment-anything)
@@ -287,7 +288,9 @@ A collection of ready-to-run Jupyter* notebooks are available to demonstrate how
287
288
-[Accelerate Inference of Sparse Transformer Models with OpenVINO and 4th Gen Intel Xeon Scalable Processors](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/116-sparsity-optimization)
288
289
289
290
### Post-Training Quantization Samples
291
+
290
292
Compact scripts demonstrating quantization and corresponding inference speed boost:
293
+
291
294
-[Post-Training Quantization of MobileNet v2 OpenVINO Model](examples/post_training_quantization/openvino/mobilenet_v2/README.md)
292
295
-[Post-Training Quantization of YOLOv8 OpenVINO Model](examples/post_training_quantization/openvino/yolov8/README.md)
293
296
-[Post-Training Quantization of Anomaly Classification OpenVINO model with control of accuracy metric](examples/post_training_quantization/openvino/quantize_with_accuracy_control/README.md)
NNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories.
313
319
314
320
### Used by
@@ -322,30 +328,39 @@ NNCF may be straightforwardly integrated into training/evaluation pipelines of t
322
328
NNCF is used as a compression backend within the renowned `transformers` repository in HuggingFace Optimum Intel.
323
329
324
330
### Git patches for third-party repository
331
+
325
332
See [third_party_integration](./third_party_integration) for examples of code modifications (Git patches and base commit IDs are provided) that are necessary to integrate NNCF into the following repositories:
For detailed installation instructions please refer to the [Installation](./docs/Installation.md) page.
330
339
331
340
NNCF can be installed as a regular PyPI package via pip:
332
-
```
341
+
342
+
```bash
333
343
pip install nncf
334
344
```
345
+
335
346
If you want to install both NNCF and the supported PyTorch version in one line, you can do this by simply running:
336
-
```
347
+
348
+
```bash
337
349
pip install nncf[torch]
338
350
```
351
+
339
352
Other viable options besides `[torch]` are `[tf]`, `[onnx]` and `[openvino]`.
340
353
341
354
NNCF is also available via [conda](https://anaconda.org/conda-forge/nncf):
342
-
```
355
+
356
+
```bash
343
357
conda install -c conda-forge nncf
344
358
```
345
359
346
-
You may also use one of the Dockerfiles in the [docker](./docker) directory to build an image with an environment already set up and ready for running NNCF [sample scripts](#Model-compression-tutorials-and-samples).
360
+
You may also use one of the Dockerfiles in the [docker](./docker) directory to build an image with an environment already set up and ready for running NNCF [sample scripts](#model-compression-tutorials-and-samples).
347
361
348
362
### System requirements
363
+
349
364
- Ubuntu\* 18.04 or later (64-bit)
350
365
- Python\* 3.7 or later
351
366
- Supported frameworks:
@@ -362,7 +377,7 @@ List of models and compression results for them can be found at our [Model Zoo p
0 commit comments