tools/downloader: align namespace (openvinotoolkit#2723)

* rename folder structure

* rame scripts

* update entry points

* fix readme

* update installation section

* update installation section

* rename package

* rename downloader folder to model_tools

* add __init__.py file

* use build_py instead build

* update readme for yolox

* update models readmes

Author: Julia Kamelina

.gitattributes

@@ -16,7 +16,7 @@
 /demos/build_demos.sh omz.package=l,m
 /demos/build_demos_msvc.bat omz.package=w
 /demos/tests/** -omz.package
-/tools/downloader/tests/** -omz.package
+/tools/model_tools/tests/** -omz.package
 
 * omz.package.component=dev
 /LICENSE omz.package.component=tools
@@ -38,6 +38,6 @@
 
 /tools/accuracy_checker/** omz.ci.job-for-change.ac
 /tools/accuracy_checker/configs/*.yml -omz.ci.job-for-change.ac
-/tools/downloader/** omz.ci.job-for-change.downloader
+/tools/model_tools/** omz.ci.job-for-change.downloader
 
 *.md omz.ci.job-for-change.documentation -omz.ci.job-for-change.ac -omz.ci.job-for-change.demos -omz.ci.job-for-change.downloader

CONTRIBUTING.md

@@ -51,8 +51,8 @@ demo|`demos/<demo_name>`<br>or<br>`demos/python_demos/<demo_name>`
 ### Tests
 
 Your PR must pass next tests:
-* Model is downloadable by the `tools/downloader/downloader.py` script. See [Configuration file](#configuration-file) for details.
-* Model is convertible by the `tools/downloader/converter.py` script. See [Model conversion](#model-conversion) for details.
+* Model is downloadable by the `tools/model_tools/downloader.py` script. See [Configuration file](#configuration-file) for details.
+* Model is convertible by the `tools/model_tools/converter.py` script. See [Model conversion](#model-conversion) for details.
 * Model is usable by demo or sample and provides adequate results. See [Demo](#demo) for details.
 * Model passes accuracy validation. See [Accuracy validation](#accuracy-validation) for details.
 
@@ -76,7 +76,7 @@ Description of the model. Must match with the description from the model [docume
 
 **`task_type`**
 
-[Model task type](tools/downloader/README.md#model-information-dumper-usage). If there is no task type of your model, add a new one to the list `KNOWN_TASK_TYPES` of the [`open_model_zoo.model_tools._common`](tools/downloader/src/open_model_zoo/model_tools/_common.py) module.
+[Model task type](tools/model_tools/README.md#model-information-dumper-usage). If there is no task type of your model, add a new one to the list `KNOWN_TASK_TYPES` of the [`openvino.model_zoo._common`](tools/model_tools/src/openvino/model_zoo/_common.py) module.
 
 **`files`**
 
@@ -135,7 +135,7 @@ Conversion parameters (learn more in the [Model conversion](#model-conversion) s
 
 **`framework`**
 
-Framework of the original model (see [here](tools/downloader/README.md#model-information-dumper-usage) for details).
+Framework of the original model (see [here](tools/model_tools/README.md#model-information-dumper-usage) for details).
 
 **`license`**
 

README.md

@@ -10,7 +10,7 @@ Intel is committed to the respect of human rights and avoiding complicity in hum
 ## Repository Components:
 * [Intel Pre-Trained Models](models/intel/index.md)
 * [Public Pre-Trained Models](models/public/index.md)
-* [Model Downloader](tools/downloader/README.md) and other automation tools
+* [Model Downloader](tools/model_tools/README.md) and other automation tools
 * [Demos](demos/README.md) that demonstrate models usage with Deep Learning Deployment Toolkit
 * [Accuracy Checker](tools/accuracy_checker/README.md) tool for models accuracy validation
 

ci/prepare-documentation.py

@@ -353,7 +353,7 @@ def main():
     add_accuracy_checker_pages(output_root, navindex_element)
 
     downloader_element = add_page(output_root, navindex_element,
-        id='omz_tools_downloader', path='tools/downloader/README.md', title='Model Downloader')
+        id='omz_tools_downloader', path='tools/model_tools/README.md', title='Model Downloader')
     downloader_element.attrib[XML_ID_ATTRIBUTE] = 'omz_tools_downloader'
 
     trained_models_group_element = add_page(output_root, navindex_element,

ci/prepare-openvino-content.py

@@ -119,7 +119,7 @@ def file_is_in_current_package(path):
     compat_symlinks = [
         ('deployment_tools/intel_models', 'open_model_zoo/models/intel'),
         ('deployment_tools/open_model_zoo/intel_models', 'models/intel'),
-        ('deployment_tools/tools/model_downloader', '../open_model_zoo/tools/downloader'),
+        ('deployment_tools/tools/model_downloader', '../open_model_zoo/tools/model_tools'),
         ('deployment_tools/inference_engine/demos', '../open_model_zoo/demos'),
     ]
 

ci/requirements-conversion.txt

@@ -27,7 +27,7 @@ defusedxml==0.7.1
 flatbuffers==1.12
     # via tensorflow
 future==0.18.2
-    # via -r tools/downloader/requirements-caffe2.in
+    # via -r tools/model_tools/requirements-caffe2.in
 gast==0.3.3
     # via tensorflow
 google-auth==1.35.0
@@ -84,8 +84,8 @@ oauthlib==3.1.1
 onnx==1.10.1
     # via
     #   -r ${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/requirements_onnx.txt
-    #   -r tools/downloader/requirements-caffe2.in
-    #   -r tools/downloader/requirements-pytorch.in
+    #   -r tools/model_tools/requirements-caffe2.in
+    #   -r tools/model_tools/requirements-pytorch.in
 opt-einsum==3.3.0
     # via tensorflow
 pillow==8.3.2
@@ -118,7 +118,7 @@ requests-oauthlib==1.3.0
 rsa==4.7.2
     # via google-auth
 scipy==1.5.4
-    # via -r tools/downloader/requirements-pytorch.in
+    # via -r tools/model_tools/requirements-pytorch.in
 six==1.15.0
     # via
     #   absl-py
@@ -140,18 +140,18 @@ tensorboard-plugin-wit==1.8.0
 tensorflow==2.4.3
     # via
     #   -r ${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/requirements_tf2.txt
-    #   -r tools/downloader/requirements-tensorflow.in
+    #   -r tools/model_tools/requirements-tensorflow.in
 tensorflow-estimator==2.4.0
     # via tensorflow
 termcolor==1.1.0
     # via tensorflow
 torch==1.8.1
     # via
-    #   -r tools/downloader/requirements-caffe2.in
-    #   -r tools/downloader/requirements-pytorch.in
+    #   -r tools/model_tools/requirements-caffe2.in
+    #   -r tools/model_tools/requirements-pytorch.in
     #   torchvision
 torchvision==0.9.1
-    # via -r tools/downloader/requirements-pytorch.in
+    # via -r tools/model_tools/requirements-pytorch.in
 typing-extensions==3.7.4.3
     # via
     #   importlib-metadata
@@ -172,7 +172,7 @@ wheel==0.37.0
 wrapt==1.12.1
     # via tensorflow
 yacs==0.1.8
-    # via -r tools/downloader/requirements-pytorch.in
+    # via -r tools/model_tools/requirements-pytorch.in
 zipp==3.5.0
     # via importlib-metadata
 

ci/requirements-downloader.txt

@@ -7,8 +7,8 @@ charset-normalizer==2.0.4
 idna==3.2
     # via requests
 pyyaml==5.4.1
-    # via -r tools/downloader/requirements.in
+    # via -r tools/model_tools/requirements.in
 requests==2.26.0
-    # via -r tools/downloader/requirements.in
+    # via -r tools/model_tools/requirements.in
 urllib3==1.26.6
     # via requests

ci/update-requirements.py

@@ -71,13 +71,13 @@ def pc(target, *sources):
     pc('ci/requirements-check-basics.txt',
        'ci/requirements-check-basics.in', 'ci/requirements-documentation.in')
     pc('ci/requirements-conversion.txt',
-        *(f'tools/downloader/requirements-{suffix}.in' for suffix in ['caffe2', 'pytorch', 'tensorflow']),
+        *(f'tools/model_tools/requirements-{suffix}.in' for suffix in ['caffe2', 'pytorch', 'tensorflow']),
         *(openvino_dir / f'deployment_tools/model_optimizer/requirements_{suffix}.txt'
             for suffix in ['caffe', 'mxnet', 'onnx', 'tf2']))
     pc('ci/requirements-demos.txt',
         'demos/requirements.txt', openvino_dir / 'python/requirements.txt')
     pc('ci/requirements-downloader.txt',
-        'tools/downloader/requirements.in')
+        'tools/model_tools/requirements.in')
     pc('ci/requirements-quantization.txt',
         'tools/accuracy_checker/requirements-core.in', 'tools/accuracy_checker/requirements.in',
         openvino_dir / 'deployment_tools/tools/post_training_optimization_toolkit/setup.py',

demos/3d_segmentation_demo/python/README.md

@@ -18,18 +18,18 @@ python3 -mpip install --user -r <omz_dir>/demos/3d_segmentation_demo/python/requ
 
 For demo input image or video files, refer to the section **Media Files Available for Demos** in the [Open Model Zoo Demos Overview](../../README.md).
 The list of models supported by the demo is in `<omz_dir>/demos/3d_segmentation_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -124,4 +124,4 @@ The demo reports
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)

demos/README.md

@@ -74,7 +74,7 @@ To run the demo applications, you can use images and videos from the media files
 
 > **NOTE:** Inference Engine HDDL plugin is available in [proprietary](https://software.intel.com/en-us/openvino-toolkit) distribution only.
 
-You can download the [Intel pre-trained models](../models/intel/index.md) or [public pre-trained models](../models/public/index.md) using the OpenVINO [Model Downloader](../tools/downloader/README.md).
+You can download the [Intel pre-trained models](../models/intel/index.md) or [public pre-trained models](../models/public/index.md) using the OpenVINO [Model Downloader](../tools/model_tools/README.md).
 
 ## Build the Demo Applications
 

demos/action_recognition_demo/python/README.md

@@ -38,18 +38,18 @@ You can change the value of `num_requests` in `action_recognition_demo.py` to fi
 
 For demo input image or video files, refer to the section **Media Files Available for Demos** in the [Open Model Zoo Demos Overview](../../README.md).
 The list of models supported by the demo is in `<omz_dir>/demos/action_recognition_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -151,4 +151,4 @@ The application uses OpenCV to display the real-time action recognition results
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)

demos/bert_named_entity_recognition_demo/python/README.md

@@ -11,18 +11,18 @@ The text is then used to search named entities.
 ## Preparing to Run
 
 The list of models supported by the demo is in `<omz_dir>/demos/bert_named_entity_recognition_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -84,7 +84,7 @@ The application reports
 ## Example Demo Cmd-Line
 
 You can use the following command to try the demo (assuming the model from the Open Model Zoo, downloaded and converted with the
-[Model Downloader](../../../tools/downloader/README.md) executed with "--name bert*"):
+[Model Downloader](../../../tools/model_tools/README.md) executed with "--name bert*"):
 
 ```sh
     python3 bert_named_entity_recognition_demo.py.py
@@ -103,5 +103,5 @@ Notice that when the original "context" (text from the url) do not fit the model
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)
 * [Benchmark C++ Sample](https://docs.openvinotoolkit.org/latest/_inference_engine_samples_benchmark_app_README.html)

demos/bert_question_answering_demo/python/README.md

@@ -11,18 +11,18 @@ The text is then used to search answers for user-provided questions.
 ## Preparing to Run
 
 The list of models supported by the demo is in `<omz_dir>/demos/bert_question_answering_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -148,5 +148,5 @@ Thus, for the long texts, the network is called multiple times. The results are
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)
 * [Benchmark C++ Sample](https://docs.openvinotoolkit.org/latest/_inference_engine_samples_benchmark_app_README.html)

demos/bert_question_answering_embedding_demo/python/README.md

@@ -20,18 +20,18 @@ If second (conventional SQuAD-tuned) Bert model is provided as well, it is used
 ## Preparing to Run
 
 The list of models supported by the demo is in `<omz_dir>/demos/bert_question_answering_embedding_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -159,5 +159,5 @@ Thus, for the long paragraph texts, the network is called multiple times as for
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)
 * [Benchmark C++ Sample](https://docs.openvinotoolkit.org/latest/_inference_engine_samples_benchmark_app_README.html)

demos/classification_demo/cpp/README.md

@@ -17,18 +17,18 @@ You can stop the demo by pressing "Esc" or "Q" button. After that, the average m
 ## Preparing to Run
 
 The list of models supported by the demo is in `<omz_dir>/demos/classification_demo/cpp/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -205,4 +205,4 @@ You can use these metrics to measure application-level performance.
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)

demos/colorization_demo/python/README.md

@@ -14,18 +14,18 @@ On startup, the application reads command-line parameters and loads one network
 
 For demo input image or video files, refer to the section **Media Files Available for Demos** in the [Open Model Zoo Demos Overview](../../README.md).
 The list of models supported by the demo is in `<omz_dir>/demos/colorization_demo/python/models.lst` file.
-This file can be used as a parameter for [Model Downloader](../../../tools/downloader/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
+This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
 
 An example of using the Model Downloader:
 
 ```sh
-python3 <omz_dir>/tools/downloader/downloader.py --list models.lst
+omz_downloader --list models.lst
 ```
 
 An example of using the Model Converter:
 
 ```sh
-python3 <omz_dir>/tools/downloader/converter.py --list models.lst
+omz_converter --list models.lst
 ```
 
 ### Supported Models
@@ -97,4 +97,4 @@ You can use both of these metrics to measure application-level performance.
 
 * [Open Model Zoo Demos](../../README.md)
 * [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
-* [Model Downloader](../../../tools/downloader/README.md)
+* [Model Downloader](../../../tools/model_tools/README.md)