|
69 | 69 | "id": "fcfc461c",
|
70 | 70 | "metadata": {},
|
71 | 71 | "source": [
|
72 |
| - "## Imports" |
| 72 | + "## Import modules" |
73 | 73 | ]
|
74 | 74 | },
|
75 | 75 | {
|
|
97 | 97 | "id": "740bfdd8",
|
98 | 98 | "metadata": {},
|
99 | 99 | "source": [
|
100 |
| - "## Load the model with AUTO device\n", |
| 100 | + "## (1) Simplify selection logic\n", |
101 | 101 | "### Default behavior of Core::compile_model API without device_name\n",
|
102 | 102 | "By default, `compile_model` API will select **AUTO** as `device_name` if no device is specified."
|
103 | 103 | ]
|
|
176 | 176 | "id": "705ce668",
|
177 | 177 | "metadata": {},
|
178 | 178 | "source": [
|
179 |
| - "## Improve first inference latency with AUTO\n", |
| 179 | + "## (2) Improve first inference latency\n", |
180 | 180 | "One of the benefits of using AUTO device selection is reducing FIL (first inference latency). FIL is model compilation time combined with first inference execution time. Using the CPU device explicitly will produce the shortest first inference latency, as the OpenVINO graph representation loads quickly on CPU using just-in-time (JIT) compilation. The challenge is with GPU devices since OpenCL graph complication to GPU-optimized kernels takes a few seconds to complete. This initialization time may be intolerable for some applications, and to avoid this delay AUTO transparently uses the CPU as the first inference device until the GPU is ready.\n",
|
181 | 181 | "### Load an Image"
|
182 | 182 | ]
|
|
298 | 298 | "tags": []
|
299 | 299 | },
|
300 | 300 | "source": [
|
301 |
| - "## Performance hint\n", |
302 |
| - "Another advantage when using AUTO device selection is the performance hint. By specifying a **LATENCY** or **THROUGHPUT** hint, AUTO optimizes the performance based on the desired metric. The **THROUGHPUT** hint delivers higher frame per second (FPS) performance than **LATENCY** hint, which delivers lower latency. The performance hints do not require any device-specific settings and they are completely portable between devices – meaning AUTO can configure the performance hint on whichever device is being used.\n", |
| 301 | + "## (3) Achieve different performance for different targets\n", |
| 302 | + "Another advantage when using AUTO device selection with the **performance hint**. By specifying a **THROUGHPUT** or **LATENCY** hint, AUTO optimizes the performance based on the desired metric. The **THROUGHPUT** hint delivers higher frame per second (FPS) performance than **LATENCY** hint, which delivers lower latency. The performance hints do not require any device-specific settings and they are completely portable between devices – meaning AUTO can configure the performance hint on whichever device is being used.\n", |
303 | 303 | "\n",
|
304 | 304 | "More information about using performance hints with AUTO: [AUTO#performance-hints](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_AUTO.html#performance-hints)\n",
|
305 | 305 | "\n",
|
|
593 | 593 | "name": "python",
|
594 | 594 | "nbconvert_exporter": "python",
|
595 | 595 | "pygments_lexer": "ipython3",
|
596 |
| - "version": "3.8.10" |
| 596 | + "version": "3.9.10" |
597 | 597 | },
|
598 | 598 | "toc-autonumbering": false,
|
599 | 599 | "toc-showmarkdowntxt": false,
|
|
0 commit comments