|
| 1 | +--- |
| 2 | +title: How to use base images |
| 3 | +description: "A guide to configuring a base image for your truss" |
| 4 | +--- |
| 5 | + |
| 6 | + |
| 7 | +Model serving enviroments will often be standardized as container images to avoid wrangling python, system, and other requirements needed to run your model on every deploy. |
| 8 | +Leverage your existing container artifacts by bringing your own base image to Truss. |
| 9 | + |
| 10 | +## Setting a base image in config.yaml |
| 11 | + |
| 12 | +To specify a base image to build a truss container image from in your `config.yaml` configure a `base_image`. |
| 13 | + |
| 14 | +```yaml config.yaml |
| 15 | +base_image: |
| 16 | + image: <image_name:tag> |
| 17 | + python_executable_path: <path-to-python> |
| 18 | +``` |
| 19 | +
|
| 20 | +where `python_executable_path` is a path to a python executable with which to run your server. |
| 21 | + |
| 22 | +## Example usage |
| 23 | + |
| 24 | +This [example truss](https://github.com/basetenlabs/truss/tree/main/examples/nemo-titanet) demonstrates how to properly configure a base image for Nvidia NeMo TitaNet: |
| 25 | + |
| 26 | +```yaml config.yaml |
| 27 | +base_image: |
| 28 | + image: nvcr.io/nvidia/nemo:23.03 |
| 29 | + python_executable_path: /usr/bin/python |
| 30 | +apply_library_patches: true |
| 31 | +bundled_packages_dir: packages |
| 32 | +data_dir: data |
| 33 | +requirements: |
| 34 | +- PySoundFile |
| 35 | +live_reload: false |
| 36 | +resources: |
| 37 | + accelerator: T4 |
| 38 | + cpu: 2500m |
| 39 | + memory: 4512Mi |
| 40 | + use_gpu: true |
| 41 | +secrets: {} |
| 42 | +spec_version: '2.0' |
| 43 | +system_packages: |
| 44 | +- python3.8-venv |
| 45 | +``` |
| 46 | + |
| 47 | +## Configuring private base images with build time secrets |
| 48 | + |
| 49 | +Secrets of the form `DOCKER_REGISTRY_<REGISTRY_URL>` will be supplied to your model build to authenticate image pulls from private container registries. |
| 50 | +For information on where to store secret values see the [secrets guide](secrets#storing-secrets-on-your-remote). |
| 51 | + |
| 52 | +For example, to configure docker credentials to a private dockerhub repository your `config.yaml` should include the following secret and placeholder: |
| 53 | + |
| 54 | +```yaml config.yaml |
| 55 | +secrets: |
| 56 | + DOCKER_REGISTRY_https://index.docker.io/v1/: null |
| 57 | +``` |
| 58 | + |
| 59 | +along with a configured Baseten secret `DOCKER_REGISTRY_https://index.docker.io/v1/` with a base64 encoded `username:password` secret value: |
| 60 | + |
| 61 | +```sh |
| 62 | +echo -n 'username:password' | base64 |
| 63 | +``` |
| 64 | + |
| 65 | +To add docker credentials for gcloud artifact registry provide an [access token](https://cloud.google.com/artifact-registry/docs/docker/authentication#token) as the secret value. |
| 66 | +For example, to configure authentication for a repository in `us-west2` your `config.yaml` should include the following secret and placeholder: |
| 67 | + |
| 68 | +```yaml config.yaml |
| 69 | +secrets: |
| 70 | + DOCKER_REGISTRY_us-west2-docker.pkg.dev: null |
| 71 | +``` |
0 commit comments