Skip to content

Commit c85ed50

Browse files
aws-diamantaws-diamant
authored
ResNet-50 tutorial cleanups
1 parent 6d37aeb commit c85ed50

File tree

1 file changed

+51
-68
lines changed

1 file changed

+51
-68
lines changed

‎docs/tensorflow-neuron/tutorial-compile-infer.md

+51-68
Original file line numberDiff line numberDiff line change
@@ -1,69 +1,73 @@
1-
# Tutorial: Using Neuron to run Resnet50 inference
1+
# Tutorial: Getting Started with TensorFlow-Neuron (ResNet-50 Tutorial)
22

33
## Steps Overview:
44

5-
1. Launch an EC2 instance for compilation and/or Infernence
6-
2. Install Neuron for Compiler and Runtime execution
7-
3. Compile on compilation server
8-
4. Execute inference on Inf1
5+
1. Launch an EC2 compilation-instance (recommended instance: c5.4xl), and an deployment instance
6+
2. Install Neuron compiler on the compilation-instace
7+
3. Compile the compute-graph on the compilation-instance, and copy the compilation artifacts to the deployment-instance
8+
4. Deploy: run inferences on the deployment-instance
99

1010
## Step 1: Launch EC2 Instance(s)
1111

12-
A typical workflow with the Neuron SDK will be to compile trained ML models on a compilation server and then distribute the artifacts to a fleet of Inf1 instances for execution. Neuron enables TensorFlow to be used for all of these steps.
12+
A typical workflow with the Neuron SDK will be to compile a trained ML model on a compilation-instance and then to distribute the artifacts to Inf1 deployment-instances, for execution.
1313

14-
1.1. Select an AMI of your choice, which may be Ubuntu 16.x, Ubuntu 18.x, Amazon Linux 2 based. To use a pre-built Deep Learning AMI, which includes all of the needed packages, see [Launching and Configuring a DLAMI](https://docs.aws.amazon.com/dlami/latest/devguide/launch-config.html)
14+
1.1. Select an AMI of your choice, which may be Ubuntu 16.x, Ubuntu 18.x, or Amazon Linux 2 based. A pre-built Deep Learning AMI is also available, which includes all of the needed packages (see https://docs.aws.amazon.com/dlami/latest/devguide/launch-config.html).
1515

16-
1.2. Select and launch an EC2 instance of your choice to compile. Launch an instance by following [EC2 instructions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html#ec2-launch-instance).
17-
* It is recommended to use c5.4xlarge or larger. For this example we will use a c5.4xlarge.
18-
* If you would like to compile and infer on the same machine, please select inf1.6xlarge.
16+
**Note:** If you choose to use Deep Learning AMI (recommended for getting started), you may skip to Step 3 below. Make sure to activate the environment of choice, e.g.
17+
```bash
18+
ubuntu:~$ source activate aws_neuron_tensorflow_p36
19+
(aws_neuron_tensorflow_p36) ubuntu:~$
20+
```
21+
22+
1.2. Select and start an EC2 compilation-instance
23+
1. For this example, we will use c5.4xlarge
24+
2. Users may choose the compile and deploy on the same instance, in which case we recommend using Inf1.6xlarge or larger
25+
26+
1.3. Select and start a deployment-instance of your choice (if not compiling and inferencing on the same instance).
27+
1. For this example, we will use Inf1.xl
1928

20-
1.3. Select and launch an Inf1 instance of your choice if not compiling and inferencing on the same instance. Launch an instance by following [EC2 instructions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html#ec2-launch-instance).
29+
## Step 2: Installations
2130

22-
## Step 2: Install Neuron Compiler and TensorFlow-Neuron On Compilation Instance
31+
### Compilation-Instance: Install Neuron Compiler and TensorFlow-Neuron
2332

24-
If using DLAMI, activate aws_neuron_tensorflow_p36 environment and skip this step.
33+
Note: this step is only required if you are not using Deep Learning AMI.
2534

26-
On the instance you are going to use for compilation, install both Neuron Compiler and TensorFlow-Neuron.
35+
On the compilation-instance, install Neuron-compiler and Tensorflow-Neuron.
36+
On the deployment-instance, install Neuron-runtime and Tensorflow-Neuron.
2737

28-
2.1. Install virtualenv if needed:
38+
#### Using Virtualenv:
39+
40+
1. Install virtualenv if needed:
2941
```bash
30-
# Ubuntu
3142
sudo apt-get update
3243
sudo apt-get -y install virtualenv
3344
```
34-
```bash
35-
# Amazon Linux 2
36-
sudo yum update
37-
sudo yum install -y python3
38-
pip3 install --user virtualenv
39-
```
40-
2.2. Setup a new Python 3.6 environment:
45+
2. Setup a new Python 3.6 environment:
4146
```bash
4247
virtualenv --python=python3.6 test_env_p36
4348
source test_env_p36/bin/activate
4449
```
45-
2.3. Modify Pip repository configurations to point to the Neuron repository.
50+
3. Modify Pip repository configurations to point to the Neuron repository.
4651
```bash
4752
tee $VIRTUAL_ENV/pip.conf > /dev/null <<EOF
4853
[global]
4954
extra-index-url = https://pip.repos.neuron.amazonaws.com
5055
EOF
5156
```
52-
2.4. Install TensorFlow-Neuron and Neuron Compiler
53-
```bash
54-
pip install tensorflow-neuron
55-
```
57+
4. Install TensorFlow-Neuron and Neuron Compiler
5658
```bash
57-
# can be skipped on inference-only instance
5859
pip install neuron-cc[tensorflow]
60+
pip install tensorflow-neuron
5961
```
6062

61-
## Step 3: Compile on Compilation Server
63+
### Deployment-Instance: Install Tensorflow-Neuron and Neuron-Runtime
64+
65+
1. Same as above to install Tensorflow Neuron.
66+
2. To install Runtime, refer to the [getting started](./../neuron-runtime/nrt_start.md) runtime guide.
6267

63-
A trained model must be compiled to Inferentia target before it can run on Inferentia.
64-
In this step we compile the Keras ResNet50 model and export it as a SavedModel which is an interchange format for TensorFlow models.
68+
## Step 3: Compile
6569

66-
3.1. Create a python script named `compile_resnet50.py` with the following content:
70+
3.1. Create a python script named `compile_resnet50.py` to compile ResNet-50. Example given below:
6771
```python
6872
import os
6973
import time
@@ -73,96 +77,75 @@ import tensorflow.neuron as tfn
7377
import tensorflow.compat.v1.keras as keras
7478
from tensorflow.keras.applications.resnet50 import ResNet50
7579
from tensorflow.keras.applications.resnet50 import preprocess_input
76-
7780
# Create a workspace
7881
WORKSPACE = './ws_resnet50'
7982
os.makedirs(WORKSPACE, exist_ok=True)
80-
8183
# Prepare export directory (old one removed)
8284
model_dir = os.path.join(WORKSPACE, 'resnet50')
8385
compiled_model_dir = os.path.join(WORKSPACE, 'resnet50_neuron')
8486
shutil.rmtree(model_dir, ignore_errors=True)
8587
shutil.rmtree(compiled_model_dir, ignore_errors=True)
86-
8788
# Instantiate Keras ResNet50 model
8889
keras.backend.set_learning_phase(0)
8990
model = ResNet50(weights='imagenet')
90-
9191
# Export SavedModel
9292
tf.saved_model.simple_save(
9393
session = keras.backend.get_session(),
9494
export_dir = model_dir,
9595
inputs = {'input': model.inputs[0]},
9696
outputs = {'output': model.outputs[0]})
97-
9897
# Compile using Neuron
9998
tfn.saved_model.compile(model_dir, compiled_model_dir)
100-
10199
# Prepare SavedModel for uploading to Inf1 instance
102100
shutil.make_archive('./resnet50_neuron', 'zip', WORKSPACE, 'resnet50_neuron')
103101
```
104-
3.2. Run the compilation script, which will take a few minutes on c5.4xlarge. At the end of script execution, the compiled SavedModel is zipped as `resnet50_neuron.zip` in local directory:
102+
103+
3.2. Run the compilation script (can take a few minutes). At the end of script execution, the compiled SavedModel is zipped as `resnet50_neuron.zip` in local directory:
105104
```bash
106105
python compile_resnet50.py
107106
```
108107
```
109-
...
110-
INFO:tensorflow:fusing subgraph neuron_op_d6f098c01c780733 with neuron-cc; log file is at /home/ubuntu/ws_resnet50/workdir/neuron_op_d6f098c01c780733/graph_def.neuron-cc.log
111-
INFO:tensorflow:Number of operations in TensorFlow session: 3978
112-
INFO:tensorflow:Number of operations after tf.neuron optimizations: 555
108+
...
109+
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
110+
INFO:tensorflow:fusing subgraph neuron_op_d6f098c01c780733 with neuron-cc
111+
INFO:tensorflow:Number of operations in TensorFlow session: 4638
112+
INFO:tensorflow:Number of operations after tf.neuron optimizations: 556
113113
INFO:tensorflow:Number of operations placed on Neuron runtime: 554
114+
INFO:tensorflow:Successfully converted ./ws_resnet50/resnet50 to ./ws_resnet50/resnet50_neuron
114115
...
115116
```
117+
116118
3.3. If not compiling and inferring on the same instance, copy the artifact to the inference server:
117119
```bash
118-
scp -i <PEM key file> ./resnet50_neuron.zip ubuntu@<instance DNS>:~/ # Ubuntu
119-
scp -i <PEM key file> ./resnet50_neuron.zip ec2-user@<instance DNS>:~/ # AML2
120+
scp -i <PEM key file> ./resnet50_neuron.zip ubuntu@<instance DNS>:~/ # is using Ubuntu-based AMI
121+
scp -i <PEM key file> ./resnet50_neuron.zip ec2-user@<instance DNS>:~/ # if using AML2-based AMI
120122
```
121123

122-
## Step 4: Install TensorFlow-Neuron and Neuron Runtime on Inference Instance
123-
124-
If using DLAMI, activate aws_neuron_tensorflow_p36 environment and skip this step.
125-
126-
On the instance you are going to use for inference, install TensorFlow-Neuron and Neuron Runtime
127-
128-
4.1. Follow Step 2 above to install TensorFlow-Neuron.
129-
* Install neuron-cc if compilation on inference instance is desired (see notes above on recommended Inf1 sizes for compilation)
130-
* Skip neuron-cc if compilation is not done on inference instance
124+
## Step 4: Deploy
131125

132-
4.2. To install Neuron Runtime, see [Getting started: Installing and Configuring Neuron-RTD](./../neuron-runtime/nrt_start.md).
133-
134-
## Step 5: Execute inference on Inf1
135-
136-
In this step we run inference on Inf1 using the model compiled in Step 3.
137-
138-
5.1. On the Inf1, create a inference Python script named `infer_resnet50.py` with the following content:
126+
4.1. On the deployment-instance (Inf1), create an inference Python script named `infer_resnet50.py` with the following content:
139127
```python
140128
import os
141-
import time
142129
import numpy as np
143130
import tensorflow as tf
144131
from tensorflow.keras.preprocessing import image
145132
from tensorflow.keras.applications import resnet50
146-
147133
# Create input from image
148134
img_sgl = image.load_img('kitten_small.jpg', target_size=(224, 224))
149135
img_arr = image.img_to_array(img_sgl)
150136
img_arr2 = np.expand_dims(img_arr, axis=0)
151137
img_arr3 = resnet50.preprocess_input(img_arr2)
152-
153138
# Load model
154139
COMPILED_MODEL_DIR = './resnet50_neuron/'
155140
predictor_inferentia = tf.contrib.predictor.from_saved_model(COMPILED_MODEL_DIR)
156-
157141
# Run inference
158142
model_feed_dict={'input': img_arr3}
159143
infa_rslts = predictor_inferentia(model_feed_dict);
160-
161144
# Display results
162145
print(resnet50.decode_predictions(infa_rslts["output"], top=5)[0])
163146
```
164147

165-
5.2. Unzip the compiled model package from Step 3, download the example image and run the inference:
148+
4.2. Unzip the model, download the example image and run the inference:
166149
```bash
167150
unzip resnet50_neuron.zip
168151
curl -O https://raw.githubusercontent.com/awslabs/mxnet-model-server/master/docs/images/kitten_small.jpg

0 commit comments

Comments
 (0)