In this quickstart guide, you will package and deploy a text classification pipeline model.
If you want to go step-by-step through these concepts and more, check out the learn model deployment tutorial for a detailed introduction to model deployment and Truss.
Install the Truss package
Install the latest version of Truss with:
pip install --upgrade truss
Create a Truss
To get started, create a Truss with the following terminal command:
truss init text-classification
When prompted, give your Truss a name like
Then, navigate to the newly created directory:
One of the two essential files in a Truss is
model/model.py. In this file, you write a
Model class: an interface between the ML model that you’re packaging and the model server that you’re running it on.
The code to load and invoke a model in a Jupyter notebook or Python script maps directly to the code used in
There are two member functions that you must implement in the
load()loads the model onto the model server. It runs exactly once when the model server is spun up or patched.
predict()handles model inference. It runs every time the model server is called.
Here’s the complete
model/model.py for the text classification model:
from transformers import pipeline
def __init__(self, **kwargs):
self._model = None
self._model = pipeline("text-classification")
def predict(self, model_input):
Add model dependencies
The other essential file in a Truss is
config.yaml, which configures the model serving environment. For a complete list of the config options, see the config reference.
config.yaml, find the line
requirements. Replace the empty list with:
No other configuration is needed.
Deploy the Truss
Truss is maintained by Baseten, which provides infrastructure for running ML models in production. We’ll use Baseten as the remote host for your model.
Get an API key
To set up the Baseten remote, you’ll need a Baseten API key. If you don’t have a Baseten account, no worries, just sign up for an account and you’ll be issued plenty of free credits to get you started.
With your Baseten API key ready to paste when prompted, you can deploy your model:
You can monitor your model deployment from your model dashboard on Baseten.
Invoke the model
After the model has finished deploying, you can invoke it from the terminal.
truss predict -d '"Truss is awesome!"'
You’ve completed the quickstart by packaging, deploying, and invoking an ML model with Truss! Next up: