Skip to content

Commit 7f5ba4a

Browse files
basetenbotbasetenbot
authored
Merge pull request #593 from basetenlabs/bump-version-0.6.2
Release 0.6.2
2 parents 3db1b78 + 804e82f commit 7f5ba4a

File tree

7 files changed

+935
-733
lines changed

7 files changed

+935
-733
lines changed

‎.github/workflows/commit_new_release_to_main.yml

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: Merge release into main
2+
3+
on:
4+
push:
5+
branches:
6+
- release
7+
8+
jobs:
9+
merge-to-main:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- name: Check out code
14+
uses: actions/checkout@v2
15+
16+
- name: Configure Git user as basetenbot
17+
run: |
18+
git config user.name "basetenbot"
19+
git config user.email "96544894+basetenbot@users.noreply.github.com"
20+
21+
- name: Fetch all branches
22+
run: |
23+
git fetch --all
24+
25+
- name: Merge release into main with priority on main changes
26+
run: |
27+
git checkout main
28+
git merge --strategy-option=ours release -m "Merge release into main prioritizing main changes"
29+
git push origin main
30+
env:
31+
GH_TOKEN: ${{ secrets.BASETENBOT_GITHUB_TOKEN }}

‎.github/workflows/create_release_pr.yml

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: Create Release PR
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
version:
7+
description: 'Version to bump to'
8+
required: true
9+
10+
jobs:
11+
bump_version:
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Checkout code
16+
uses: actions/checkout@v2
17+
18+
- name: Install poetry
19+
run: |
20+
curl -sSL https://install.python-poetry.org | python3 -
21+
22+
- name: Bump version in pyproject.toml
23+
run: |
24+
poetry version $INPUT_VERSION
25+
env:
26+
INPUT_VERSION: ${{ github.event.inputs.version }}
27+
28+
29+
- name: Commit changes
30+
run: |
31+
git config --local user.email "96544894+basetenbot@users.noreply.github.com"
32+
git config --local user.name "basetenbot"
33+
git add pyproject.toml
34+
git commit -m "Bump version to $INPUT_VERSION"
35+
env:
36+
INPUT_VERSION: ${{ github.event.inputs.version }}
37+
38+
# TODO: Also push changes to main
39+
- name: Push changes to new branch
40+
run: |
41+
git push origin HEAD:refs/heads/bump-version-${{ github.event.inputs.version }}
42+
env:
43+
GH_TOKEN: ${{ secrets.BASETENBOT_GITHUB_TOKEN }}
44+
45+
- name: Make PR
46+
run: |
47+
CURR_VERSION=$(curl https://pypi.org/pypi/truss/json | jq ".info.version")
48+
PR_BODY="Updating Truss from [$CURR_VERSION](https://pypi.org/pypi/truss/json) to $INPUT_VERSION."
49+
PR_URL=$(gh pr create --base release --head refs/heads/bump-version-$INPUT_VERSION --title "Release $INPUT_VERSION" --body "$PR_BODY")
50+
gh pr merge $PR_URL --auto --merge
51+
env:
52+
INPUT_VERSION: ${{ github.event.inputs.version }}
53+
GH_TOKEN: ${{ secrets.BASETENBOT_GITHUB_TOKEN }}

‎README.md

+60-22
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,9 @@
1313

1414
See Trusses for popular models including:
1515

16-
* 🦅 [Falcon 40B](https://github.com/basetenlabs/falcon-40b-truss)
17-
* 🧙 [WizardLM](https://github.com/basetenlabs/wizardlm-truss)
18-
* 🎨 [Stable Diffusion](https://github.com/basetenlabs/stable-diffusion-truss)
19-
* 🗣 [Whisper](https://github.com/basetenlabs/whisper-truss)
16+
* 🦙 [Llama 2 7B](https://github.com/basetenlabs/truss-examples/tree/main/llama-2-7b-chat) ([13B](https://github.com/basetenlabs/truss-examples/tree/main/llama-2-13b-chat)) ([70B](https://github.com/basetenlabs/truss-examples/tree/main/llama-2-70b-chat))
17+
* 🎨 [Stable Diffusion XL](https://github.com/basetenlabs/truss-examples/tree/main/stable-diffusion-xl-1.0)
18+
* 🗣 [Whisper](https://github.com/basetenlabs/truss-examples/tree/main/whisper-truss)
2019

2120
and [dozens more examples](examples/).
2221

@@ -36,61 +35,100 @@ As a quick example, we'll package a [text classification pipeline](https://huggi
3635

3736
To get started, create a Truss with the following terminal command:
3837

39-
```
38+
```sh
4039
truss init text-classification
4140
```
4241

43-
This will create an empty Truss at `./text-classification`.
42+
When prompted, give your Truss a name like `Text classification`.
43+
44+
Then, navigate to the newly created directory:
45+
46+
```sh
47+
cd text-classification
48+
```
4449

4550
### Implement the model
4651

47-
The model serving code goes in `./text-classification/model/model.py` in your newly created Truss.
52+
One of the two essential files in a Truss is `model/model.py`. In this file, you write a `Model` class: an interface between the ML model that you're packaging and the model server that you're running it on.
53+
54+
There are two member functions that you must implement in the `Model` class:
55+
56+
* `load()` loads the model onto the model server. It runs exactly once when the model server is spun up or patched.
57+
* `predict()` handles model inference. It runs every time the model server is called.
58+
59+
Here's the complete `model/model.py` for the text classification model:
4860

4961
```python
50-
from typing import List
5162
from transformers import pipeline
5263

5364

5465
class Model:
55-
def __init__(self, **kwargs) -> None:
66+
def __init__(self, **kwargs):
5667
self._model = None
5768

5869
def load(self):
5970
self._model = pipeline("text-classification")
6071

61-
def predict(self, model_input: str) -> List:
72+
def predict(self, model_input):
6273
return self._model(model_input)
6374
```
6475

65-
There are two functions to implement:
66-
67-
* `load()` runs once when the model is spun up and is responsible for initializing `self._model`
68-
* `predict()` runs each time the model is invoked and handles the inference. It can use any JSON-serializable type as input and output.
69-
7076
### Add model dependencies
7177

72-
The pipeline model relies on Transformers and PyTorch. These dependencies must be specified in the Truss config.
78+
The other essential file in a Truss is `config.yaml`, which configures the model serving environment. For a complete list of the config options, see [the config reference](https://truss.baseten.co/reference/config).
7379

74-
In `./text-classification/config.yaml`, find the line `requirements`. Replace the empty list with:
80+
The pipeline model relies on [Transformers](https://huggingface.co/docs/transformers/index) and [PyTorch](https://pytorch.org/). These dependencies must be specified in the Truss config.
81+
82+
In `config.yaml`, find the line `requirements`. Replace the empty list with:
7583

7684
```yaml
7785
requirements:
7886
- torch==2.0.1
7987
- transformers==4.30.0
8088
```
8189
82-
No other configuration needs to be changed.
90+
No other configuration is needed.
8391
8492
## Deployment
8593
86-
You can deploy a Truss to your [Baseten](https://baseten.co) account with:
94+
Truss is maintained by [Baseten](https://baseten.co), which provides infrastructure for running ML models in production. We'll use Baseten as the remote host for your model.
8795
88-
```
89-
cd ./text-classification
96+
Other remotes are coming soon, starting with AWS SageMaker.
97+
98+
### Get an API key
99+
100+
To set up the Baseten remote, you'll need a [Baseten API key](https://app.baseten.co/settings/account/api_keys). If you don't have a Baseten account, no worries, just [sign up for an account](https://app.baseten.co/signup/) and you'll be issued plenty of free credits to get you started.
101+
102+
### Run `truss push`
103+
104+
With your Baseten API key ready to paste when prompted, you can deploy your model:
105+
106+
```sh
90107
truss push
91108
```
92109

93-
Truss will support other remotes soon, starting with AWS SageMaker.
110+
You can monitor your model deployment from [your model dashboard on Baseten](https://app.baseten.co/models/).
111+
112+
### Invoke the model
113+
114+
After the model has finished deploying, you can invoke it from the terminal.
115+
116+
**Invocation**
117+
118+
```sh
119+
truss predict -d '"Truss is awesome!"'
120+
```
121+
122+
**Response**
123+
124+
```json
125+
[
126+
{
127+
"label": "POSITIVE",
128+
"score": 0.999873161315918
129+
}
130+
]
131+
```
94132

95133
## Truss contributors
96134

0 commit comments

Comments
 (0)