documentation updates (openvinotoolkit#623)

Author: dtrawins

README.md

@@ -105,10 +105,14 @@ Follow a [contributor guide](docs/contributing.md) and a [developer guide](docs/
 
 * [RESTful API](https://restfulapi.net/)
 
+* [Benchmarking results](https://docs.openvinotoolkit.org/latest/openvino_docs_performance_benchmarks_ovms.html)
+
 * [Speed and Scale AI Inference Operations Across Multiple Architectures](https://techdecoded.intel.io/essentials/speed-and-scale-ai-inference-operations-across-multiple-architectures/?elq_cid=3646480_ts1607680426276&erpm_id=6470692_ts1607680426276) - webinar recording
 
 * [What is new in OpenVINO Model Server C++](https://www.intel.com/content/www/us/en/artificial-intelligence/posts/whats-new-openvino-model-server.html)
 
+* [Capital Health Improves Stroke Care with AI](https://www.intel.co.uk/content/www/uk/en/customer-spotlight/stories/capital-health-ai-customer-story.html) - use case example
+
 ## Contact
 
 Submit Github issue to ask question, request a feature or report a bug.

deploy/README.md

@@ -257,7 +257,7 @@ release "ovms-app" uninstalled
 | aws_region      | S3 storage secret key, use it with S3 storage for models |  | - |
 | aws_secret_access_key      | S3 storage secret key, use it with S3 storage for models |  | - |
 | s3_compat_api_endpoint      | S3 compatibility api endpoint, use it with Minio storage for models |  | - |
-| azure_storage_connection_string   | S3 compatibility api endpoint, use it with Minio storage for models |  | - |
+| azure_storage_connection_string   | connection string to the Azure Storage authentication account, use it with Azure storage for models |  | - |
 | log_level      | OVMS log level, one of ERROR,INFO,DEBUG|  |  INFO |
 | service_type      | k8s service type |  | LoadBalancer |
 | resources      | compute resource limits |  | All CPU and memory on the node |

example_client/README.md

@@ -156,47 +156,47 @@ Start processing:
 	Images numpy path: imgs.npy
 	Images in shape: (10, 3, 224, 224)
 
-Iteration 1; Processing time: 55.45 ms; speed 18.03 fps
+Iteration 1; Processing time: 26.54 ms; speed 37.67 fps
 imagenet top results in a single batch:
-	 0 warplane, military plane 895 ; Incorrect match. Should be 404 airliner
-Iteration 2; Processing time: 71.97 ms; speed 13.89 fps
+	 0 airliner 404 ; Correct match.
+Iteration 2; Processing time: 22.23 ms; speed 44.99 fps
 imagenet top results in a single batch:
 	 0 Arctic fox, white fox, Alopex lagopus 279 ; Correct match.
-Iteration 3; Processing time: 69.82 ms; speed 14.32 fps
+Iteration 3; Processing time: 21.72 ms; speed 46.03 fps
 imagenet top results in a single batch:
 	 0 bee 309 ; Correct match.
-Iteration 4; Processing time: 68.95 ms; speed 14.50 fps
+Iteration 4; Processing time: 20.71 ms; speed 48.28 fps
 imagenet top results in a single batch:
 	 0 golden retriever 207 ; Correct match.
-Iteration 5; Processing time: 49.82 ms; speed 20.07 fps
+Iteration 5; Processing time: 20.53 ms; speed 48.71 fps
 imagenet top results in a single batch:
 	 0 gorilla, Gorilla gorilla 366 ; Correct match.
-Iteration 6; Processing time: 56.90 ms; speed 17.58 fps
+Iteration 6; Processing time: 20.37 ms; speed 49.08 fps
 imagenet top results in a single batch:
 	 0 magnetic compass 635 ; Correct match.
-Iteration 7; Processing time: 122.50 ms; speed 8.16 fps
+Iteration 7; Processing time: 20.97 ms; speed 47.68 fps
 imagenet top results in a single batch:
 	 0 peacock 84 ; Correct match.
-Iteration 8; Processing time: 50.65 ms; speed 19.74 fps
+Iteration 8; Processing time: 22.82 ms; speed 43.83 fps
 imagenet top results in a single batch:
 	 0 pelican 144 ; Correct match.
-Iteration 9; Processing time: 56.45 ms; speed 17.71 fps
+Iteration 9; Processing time: 22.16 ms; speed 45.13 fps
 imagenet top results in a single batch:
 	 0 snail 113 ; Correct match.
-Iteration 10; Processing time: 58.95 ms; speed 16.96 fps
+Iteration 10; Processing time: 21.17 ms; speed 47.24 fps
 imagenet top results in a single batch:
 	 0 zebra 340 ; Correct match.
 
 processing time for all iterations
-average time: 65.40 ms; average speed: 15.29 fps
-median time: 57.00 ms; median speed: 17.54 fps
-max time: 122.00 ms; max speed: 8.20 fps
-min time: 49.00 ms; min speed: 20.41 fps
-time percentile 90: 76.10 ms; speed percentile 90: 13.14 fps
-time percentile 50: 57.00 ms; speed percentile 50: 17.54 fps
-time standard deviation: 20.25
-time variance: 410.04
-Classification accuracy: 90.00
+average time: 21.40 ms; average speed: 46.73 fps
+median time: 21.00 ms; median speed: 47.62 fps
+max time: 26.00 ms; min speed: 38.46 fps
+min time: 20.00 ms; max speed: 50.00 fps
+time percentile 90: 22.40 ms; speed percentile 90: 44.64 fps
+time percentile 50: 21.00 ms; speed percentile 50: 47.62 fps
+time standard deviation: 1.74
+time variance: 3.04
+Classification accuracy: 100.00
 
 ```
 
@@ -235,38 +235,41 @@ python jpeg_classification.py --grpc_port 9000 --input_name data --output_name p
 	Model name: resnet
 	Images list file: input_images.txt
 
+Start processing:
+	Model name: resnet
+	Images list file: input_images.txt
 images/airliner.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 20.00 ms; speed 2.00 fps 49.03
-Detected: 404  Should be: 404
+Processing time: 23.40 ms; speed 42.73 fps
+	 1 airliner 404 ; Correct match.
 images/arctic-fox.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 22.00 ms; speed 2.00 fps 45.2
-Detected: 279  Should be: 279
+Processing time: 21.09 ms; speed 47.42 fps
+	 2 Arctic fox, white fox, Alopex lagopus 279 ; Correct match.
 images/bee.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 14.00 ms; speed 2.00 fps 69.12
-Detected: 309  Should be: 309
+Processing time: 21.04 ms; speed 47.52 fps
+	 3 bee 309 ; Correct match.
 images/golden_retriever.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 22.00 ms; speed 2.00 fps 45.68
-Detected: 207  Should be: 207
+Processing time: 20.03 ms; speed 49.92 fps
+	 4 golden retriever 207 ; Correct match.
 images/gorilla.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 18.00 ms; speed 2.00 fps 56.02
-Detected: 366  Should be: 366
+Processing time: 21.95 ms; speed 45.57 fps
+	 5 gorilla, Gorilla gorilla 366 ; Correct match.
 images/magnetic_compass.jpeg (1, 3, 224, 224) ; data range: 0.0 : 247.0
-Processing time: 20.00 ms; speed 2.00 fps 50.78
-Detected: 635  Should be: 635
+Processing time: 21.51 ms; speed 46.48 fps
+	 6 magnetic compass 635 ; Correct match.
 images/peacock.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 21.00 ms; speed 2.00 fps 47.1
-Detected: 84  Should be: 84
+Processing time: 20.81 ms; speed 48.05 fps
+	 7 peacock 84 ; Correct match.
 images/pelican.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 24.00 ms; speed 2.00 fps 41.32
-Detected: 144  Should be: 144
+Processing time: 21.90 ms; speed 45.66 fps
+	 8 pelican 144 ; Correct match.
 images/snail.jpeg (1, 3, 224, 224) ; data range: 0.0 : 248.0
-Processing time: 20.00 ms; speed 2.00 fps 49.16
-Detected: 113  Should be: 113
+Processing time: 22.38 ms; speed 44.68 fps
+	 9 snail 113 ; Correct match.
 images/zebra.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
-Processing time: 21.00 ms; speed 2.00 fps 48.16
-Detected: 340  Should be: 340
+Processing time: 22.22 ms; speed 45.00 fps
+	 10 zebra 340 ; Correct match.
 Overall accuracy= 100.0 %
-Average latency= 19.8 ms
+Average latency= 21.2 ms
 ```
 ### Multiple input example for HDDL
 

example_client/jpeg_classification.py

@@ -16,6 +16,7 @@
 
 import grpc
 import numpy as np
+import classes
 from tensorflow import make_tensor_proto, make_ndarray, make_tensor_proto
 import datetime
 import argparse
@@ -109,10 +110,14 @@ def getJpeg(path, size, rgb_image=0):
     # for object classification models show imagenet class
     print('Processing time: {:.2f} ms; speed {:.2f} fps'.format(round(duration, 2), round(1000 / duration, 2)))
     ma = np.argmax(nu)
+    mark_message = ""
     if int(label) == ma:
         matched += 1
+        mark_message = "; Correct match."
+    else:
+        mark_message = "; Incorrect match. Should be {} {}".format(lb[i], classes.imagenet_classes[lb[i]] )
     i += 1
-    print("Detected:", ma, " Should be:", label)
+    print("\t",i, classes.imagenet_classes[ma],ma, mark_message)
 
 latency = np.average(processing_times)
 accuracy = matched/i

extras/ovms-operator/tests/README.md

@@ -26,14 +26,16 @@ Prerequisites:
 export REGISTRY=quay.io
 export IMG_BUNDLE=$REGISTRY/ovms-operator-bundle:latest
 export IMG_CATALOG=$REGISTRY/test-catalog:latest
-git clone -b ovms-operator https://github.com/community-operators
+git clone https://github.com/operator-framework/community-operators
 cd community-operators/upstream-community-operators/ovms-operator/0.1.0
 
 # edit manifests/ovms-operator.clusterserviceversion.yaml and set image: to point to the test operator image
 
 podman build -t $IMG_BUNDLE .
-podman push $IMG_BUNDLE
-opm index add --bundles $IMG_BUNDLE --from-index quay.io/operatorhubio/catalog:latest --tag $IMG_CATALOG
+podman build -t $IMG_BUNDLE .
+podman build -t $IMG_BUNDLE .
+or
+opm index add --bundles $IMG_BUNDLE --from-index registry.redhat.io/redhat/community-operator-index:v4.7 -p podman --tag $IMG_CATALOG
 podman push $IMG_CATALOG
 ```