Base images are used to create Docker images for containers. Truss uses customized base images to:
- Reduce Docker image build times for model serving
- Reduce container startup time
- Re-use common parts of containers
For a general background on base images, learn more in Docker’s official documentation.
Generating base images for Truss
This script can be used to generate and push base images to Docker Hub. The script allows a lot of control over which images to generate. To learn about all options, run:
This example generates a base image for a model serving Truss with live reload and GPU support. The Truss uses Python 3.9 and has a version tag of
v0.2.2. The generated image is named
poetry run bin/generate_base_images.py --live-reload y --use-gpu y --python-version 3.9 --version-tag v0.2.2 --job-type server
This example generates and pushes to Docker Hub all images with the version tag
v0.2.2. This requires access to a specific Docker Hub account, which is not available publicly.
poetry run bin/generate_base_images.py --version-tag v0.2.2 --push
Working with base images in Truss
This section will guide you through testing and releasing new base images for Truss.
Testing base images
Building every base image locally to test on your local Docker takes a long time. Be selective about which base images you need to build and test based on your changes. For example, you might only need to build serving images for Python 3.9 without GPU support.
Also, we recommend using a custom version tag to keep things clean while testing.
Publishing base images
New base images are only published when changes are released that affect base images. This prevents proliferation of identical images with different versions.
When publishing base images, their semantic versioning must match the version of Truss they were built from, even if that skips versions from the existing base images. This makes it easier to track where the code came from.
To publish a base image:
- Increment Truss version in pyproject.toml
/workspaces/truss/truss/contexts/image_builder/util.py::TRUSS_BASE_IMAGE_VERSION_TAGto match this version
- Create a PR and get it reviewed.
- Merge the PR.
- Upon merge, a Github Action will run that generates all the necessary base images
- Make sure integration tests pass.
Releasing Truss versions with new base images
If and only if your changes to Truss require a new base image, follow these steps before publishing a new version to PyPi:
- Generate your new base images with the
TRUSS_BASE_IMAGE_VERSION_TAGto match the new Truss version so that the new base images are used.
- Push the new base images before running integration tests. (The new images are not used for integration tests until the Truss library and context builder are published)
- Increment the Truss version and publish to PyPi as normal.