Skip to content

Commit c5b89a6

Browse files
Remove Inputs Key from Truss Templates (#244)
* the actual change * oops rm extra files * create docs * fix up all tests for new approach * update select examples to demonstrate input-free approach * tests and pre/post processing * rm any
1 parent 7d73c69 commit c5b89a6

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+455
-396
lines changed

README.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ rfc.fit(data_x, data_y)
7171
tr = truss.create(rfc, target_directory="iris_rfc_truss")
7272

7373
# Serve a prediction from the model
74-
tr.predict({"inputs": [[0, 0, 0, 0]]})
74+
tr.predict([[0, 0, 0, 0]])
7575
```
7676

7777
### Package your model
@@ -80,6 +80,7 @@ The `truss.create()` command can be used with any supported framework:
8080

8181
* [Hugging Face](https://truss.baseten.co/create/huggingface)
8282
* [LightGBM](https://truss.baseten.co/create/lightgbm)
83+
* [MLflow](https://truss.baseten.co/create/mlflow)
8384
* [PyTorch](https://truss.baseten.co/create/pytorch)
8485
* [scikit-learn](https://truss.baseten.co/create/sklearn)
8586
* [Tensorflow](https://truss.baseten.co/create/tensorflow)
@@ -98,7 +99,7 @@ truss run-image iris_rfc_truss
9899
Then, as long as the container is running, you can invoke the model as an API as follows:
99100

100101
```
101-
curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '{"inputs": [[0, 0, 0, 0]]}'
102+
curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '[[0, 0, 0, 0]]'
102103
```
103104

104105
### Configure your model for deployment

docs/create/huggingface.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ If you're using a Jupyter notebook, add a line to install the `transformers` and
1313
{% hint style="warning" %}
1414
Truss officially supports `transformers` version 4.21.0 or higher. Especially if you're using an online notebook environment like Google Colab or a bundle of packages like Anaconda, ensure that the version you are using is supported. If it's not, use the `--upgrade` flag and pip will install the most recent version.
1515
{% endhint %}
16+
1617
### Create an in-memory model
1718

1819
This is the part you want to replace with your own code. Using a Hugging Face transformer, build a machine learning model and keep it in-memory. In this example we're using [bert-base-uncased](https://huggingface.co/bert-base-uncased), which will fill in the missing word in a sentence.
@@ -44,7 +45,7 @@ Check the target directory to see your new Truss!
4445
To get a prediction from the Truss, try running:
4546

4647
```python
47-
tr.predict({"inputs": ["Donatello is a teenage mutant [MASK] turtle"]})
48+
tr.predict("Donatello is a teenage mutant [MASK] turtle")
4849
```
4950

5051
For more on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/lightgbm.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Check the target directory to see your new Truss!
6666
To get a prediction from the Truss, try running:
6767

6868
```python
69-
tr.predict({"inputs": [[0, 0, 0, 0, 0, 0]]})
69+
tr.predict([[0, 0, 0, 0, 0, 0]])
7070
```
7171

7272
For more on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/mlflow.md

+1-2
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,7 @@ To get a prediction from the Truss, try running:
5353

5454
```python
5555
data = np.array([-4, 1, 0, 10, -2, 1]).reshape(-1, 1)
56-
predictions = tr.predict({"inputs": data})
57-
print(predictions)
56+
tr.predict(data)
5857
```
5958

6059
For more on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/pytorch.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ inputs = datasets.MNIST("../data", train=False, transform=transform)
165165
dataset = torch.utils.data.DataLoader(inputs, batch_size=1)
166166

167167
import numpy as np
168-
print(tr.predict({"inputs": np.array(next(iter(dataset))[0])}))
168+
print(tr.predict(np.array(next(iter(dataset))[0])))
169169
```
170170

171171
For information on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/sklearn.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ Check the target directory to see your new Truss!
4545
To get a prediction from the Truss, try running:
4646

4747
```python
48-
tr.predict({"inputs": [[0, 0, 0, 0]]})
48+
tr.predict([[0, 0, 0, 0]])
4949
```
5050

5151
For more on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/tensorflow.md

+27-12
Original file line numberDiff line numberDiff line change
@@ -44,19 +44,24 @@ Check the target directory to see your new Truss!
4444

4545
### Serve the model
4646

47-
The TensorFlow model requires [pre- and post-processing functions](../develop/processing.md) to run. These functions go in `model/model.py`:
47+
In your newly created Truss, open `model/model.py` and add pre- and post-processing functions as follows.
48+
49+
First, add the following imports at the top of the file:
50+
4851

4952
```python
50-
#Preprocess and Postprocess Functions
5153
import requests
5254
import tempfile
53-
import numpy as np
54-
55+
import tensorflow as tf
5556
from scipy.special import softmax
57+
```
58+
59+
Then, update the pre-processing function to:
5660

57-
def preprocess(url):
61+
```python
62+
def preprocess(self, model_input: Any) -> Any:
5863
"""Preprocess step for ResNet"""
59-
request = requests.get(url)
64+
request = requests.get(model_input)
6065
with tempfile.NamedTemporaryFile() as f:
6166
f.write(request.content)
6267
f.seek(0)
@@ -65,22 +70,32 @@ def preprocess(url):
6570
tf.image.resize([input_image], (224, 224))
6671
)
6772
return np.array(preprocessed_image)
73+
```
74+
75+
Finally, update the post-processing function to:
6876

69-
def postprocess(predictions, k=5):
77+
```python
78+
def postprocess(self, model_output: Dict, k=5) -> Dict:
7079
"""Post process step for ResNet"""
71-
class_predictions = predictions[0]
80+
class_predictions = model_output["predictions"][0]
7281
LABELS = requests.get(
73-
'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'
74-
).text.split('\n')
82+
"https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt"
83+
).text.split("\n")
7584
class_probabilities = softmax(class_predictions)
7685
top_probability_indices = class_probabilities.argsort()[::-1][:k].tolist()
77-
return {LABELS[index]: 100 * class_probabilities[index].round(3) for index in top_probability_indices}
86+
return {
87+
LABELS[index]: 100 * class_probabilities[index].round(3)
88+
for index in top_probability_indices
89+
}
7890
```
7991

8092
With these functions in place, you can invoke the model and pass it a URL, as in:
8193

8294
```python
83-
tr.predict({"inputs": "https://github.com/pytorch/hub/raw/master/images/dog.jpg"})
95+
from truss import load
96+
97+
tr = load("tensorflow_truss")
98+
tr.predict("https://github.com/pytorch/hub/raw/master/images/dog.jpg")
8499
```
85100

86101
For information on running the Truss locally, see [local development](../develop/localhost.md).

docs/create/xgboost.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ Check the target directory to see your new Truss!
6161
To get a prediction from the Truss, try running:
6262

6363
```python
64-
tr.predict({"inputs": [[0, 0, 0, 0, 0, 0]]})
64+
tr.predict([[0, 0, 0, 0, 0, 0]])
6565
```
6666

6767
For more on running the Truss locally, see [local development](../develop/localhost.md).

docs/deploy/aws.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -91,5 +91,5 @@ Now that you're task is running, you can make requests to your model! To get the
9191
If you've been following along with the model above, you can use the snippet below to make a sample request.
9292

9393
```
94-
curl -H 'Content-Type: application/json' -d '{"inputs": [[0,0,0,0]]}' -X POST [CONTAINER_LINK]:80/v1/models/model:predict
94+
curl -H 'Content-Type: application/json' -d '[[0,0,0,0]]' -X POST [CONTAINER_LINK]:80/v1/models/model:predict
9595
```

docs/develop/localhost.md

+6-6
Original file line numberDiff line numberDiff line change
@@ -27,15 +27,15 @@ tr = truss.load("path_to_my_truss")
2727
From there, you can invoke the Truss to serve the model in your Python environment. Just run:
2828

2929
```python
30-
tr.predict({"inputs": [[0, 0, 0, 0]]})
30+
tr.predict([[0, 0, 0, 0]])
3131
```
3232

3333
### Command line interface
3434

3535
Alternately, you can run your Truss from the command line.
3636

3737
```
38-
truss predict path_to_my_truss '{"inputs": [[0, 0, 0, 0]]}'
38+
truss predict path_to_my_truss '[[0, 0, 0, 0]]'
3939
```
4040

4141
You can also specify examples for the model and run them instead. It's much easier to express request data in the example file. Running the example provides for a good dev loop.
@@ -51,7 +51,7 @@ truss run-image path_to_my_truss
5151
Then, as long as the container is running, you can invoke the model as an API as follows:
5252

5353
```
54-
curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '{"inputs": [[0, 0, 0, 0]]}'
54+
curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '{[[0, 0, 0, 0]]'
5555
```
5656

5757
## Setting up local dev
@@ -69,7 +69,7 @@ This is the standard way to run a Truss. It creates a Docker image, runs the Tru
6969
To run in Docker:
7070

7171
```python
72-
tr.docker_predict({"inputs": [[0, 0, 0, 0]]})
72+
tr.docker_predict([[0, 0, 0, 0]])
7373
```
7474

7575
### Faster dev loop with live reload
@@ -112,11 +112,11 @@ Unlike Docker image, this mechanism requires that you already have the right Pyt
112112
In the Python environment, get a prediction without Docker by running:
113113

114114
```python
115-
tr.predict({"inputs": [[0, 0, 0, 0]]})
115+
tr.predict([[0, 0, 0, 0]])
116116
```
117117

118118
Or in the command line, run:
119119

120120
```python
121-
truss predict --run-local path_to_my_truss '{"inputs": [[0, 0, 0, 0]]}'
121+
truss predict --run-local path_to_my_truss '[[0, 0, 0, 0]]'
122122
```

docs/develop/processing.md

+10-7
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,9 @@ import requests
1919
import tempfile
2020
import numpy as np
2121

22-
def preprocess(url):
22+
def preprocess(self, model_input: Any) -> Any:
2323
"""Preprocess step for ResNet"""
24-
request = requests.get(url)
24+
request = requests.get(model_input)
2525
with tempfile.NamedTemporaryFile() as f:
2626
f.write(request.content)
2727
f.seek(0)
@@ -41,13 +41,16 @@ Here is a post-processing function from the same [TensorFlow example](../create/
4141
```python
4242
from scipy.special import softmax
4343

44-
def postprocess(predictions, k=5):
44+
def postprocess(self, model_output: Dict, k=5) -> Dict:
4545
"""Post process step for ResNet"""
46-
class_predictions = predictions[0]
46+
class_predictions = model_output["predictions"][0]
4747
LABELS = requests.get(
48-
'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'
49-
).text.split('\n')
48+
"https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt"
49+
).text.split("\n")
5050
class_probabilities = softmax(class_predictions)
5151
top_probability_indices = class_probabilities.argsort()[::-1][:k].tolist()
52-
return {LABELS[index]: 100 * class_probabilities[index].round(3) for index in top_probability_indices}
52+
return {
53+
LABELS[index]: 100 * class_probabilities[index].round(3)
54+
for index in top_probability_indices
55+
}
5356
```

docs/notebooks/huggingface_example.ipynb

+12-4
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
"outputs": [],
6060
"source": [
6161
"# Serve a prediction from the model\n",
62-
"tr.predict({\"inputs\": [\"Donatello is a teenage mutant [MASK] turtle\"]})"
62+
"tr.predict(\"Donatello is a teenage mutant [MASK] turtle\")"
6363
]
6464
}
6565
],
@@ -69,17 +69,25 @@
6969
"provenance": []
7070
},
7171
"kernelspec": {
72-
"display_name": "Python 3",
72+
"display_name": "truss-qbHCcytf-py3.9",
7373
"language": "python",
7474
"name": "python3"
7575
},
7676
"language_info": {
77+
"codemirror_mode": {
78+
"name": "ipython",
79+
"version": 3
80+
},
81+
"file_extension": ".py",
82+
"mimetype": "text/x-python",
7783
"name": "python",
78-
"version": "3.8.9 (default, Apr 13 2022, 08:48:06) \n[Clang 13.1.6 (clang-1316.0.21.2.5)]"
84+
"nbconvert_exporter": "python",
85+
"pygments_lexer": "ipython3",
86+
"version": "3.9.16"
7987
},
8088
"vscode": {
8189
"interpreter": {
82-
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
90+
"hash": "71eaf67bb821e55ec2e9ffe63f35d72d7631d0e358c20043383b5ebba2b8daf8"
8391
}
8492
}
8593
},

docs/notebooks/lightgbm_example.ipynb

+18-3
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@
8181
},
8282
"outputs": [],
8383
"source": [
84-
"tr.predict({\"inputs\": [[0, 0, 0, 0, 0, 0]]})"
84+
"tr.predict([[0, 0, 0, 0, 0, 0]])"
8585
]
8686
}
8787
],
@@ -92,11 +92,26 @@
9292
"provenance": []
9393
},
9494
"kernelspec": {
95-
"display_name": "Python 3",
95+
"display_name": "truss-qbHCcytf-py3.9",
96+
"language": "python",
9697
"name": "python3"
9798
},
9899
"language_info": {
99-
"name": "python"
100+
"codemirror_mode": {
101+
"name": "ipython",
102+
"version": 3
103+
},
104+
"file_extension": ".py",
105+
"mimetype": "text/x-python",
106+
"name": "python",
107+
"nbconvert_exporter": "python",
108+
"pygments_lexer": "ipython3",
109+
"version": "3.9.16"
110+
},
111+
"vscode": {
112+
"interpreter": {
113+
"hash": "71eaf67bb821e55ec2e9ffe63f35d72d7631d0e358c20043383b5ebba2b8daf8"
114+
}
100115
}
101116
},
102117
"nbformat": 4,

docs/notebooks/mlflow_example.ipynb

+12-5
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,7 @@
9999
"# Invoke the MLflow model\n",
100100
"\n",
101101
"data = np.array([-4, 1, 0, 10, -2, 1]).reshape(-1, 1)\n",
102-
"predictions = tr.predict({\"inputs\": data})\n",
103-
"print(predictions)"
102+
"tr.predict(data)"
104103
]
105104
}
106105
],
@@ -109,17 +108,25 @@
109108
"provenance": []
110109
},
111110
"kernelspec": {
112-
"display_name": "Python 3.8.9 64-bit",
111+
"display_name": "truss-qbHCcytf-py3.9",
113112
"language": "python",
114113
"name": "python3"
115114
},
116115
"language_info": {
116+
"codemirror_mode": {
117+
"name": "ipython",
118+
"version": 3
119+
},
120+
"file_extension": ".py",
121+
"mimetype": "text/x-python",
117122
"name": "python",
118-
"version": "3.8.9"
123+
"nbconvert_exporter": "python",
124+
"pygments_lexer": "ipython3",
125+
"version": "3.9.16"
119126
},
120127
"vscode": {
121128
"interpreter": {
122-
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
129+
"hash": "71eaf67bb821e55ec2e9ffe63f35d72d7631d0e358c20043383b5ebba2b8daf8"
123130
}
124131
}
125132
},

0 commit comments

Comments
 (0)