Your model server may need to use access tokens, API keys, passwords, or other secret values. Truss gives you everything you need to use secrets securely.
Setting secrets in
If your model needs a secret, first add its name in
config.yaml with a placeholder value:
secrets: hf_access_token: null
Never set the actual value of a secret in the
config.yaml file. Only put secret values in secure places, like the Baseten workspace secret manager.
Using secrets in
Secrets are passed to your
Model class as a keyword argument in
init. They can be accessed with:
def __init__(self, **kwargs): self._secrets = kwargs["secrets"]
You can then use the
self._secrets dictionary in the
def load(self): self._model = pipeline( "fill-mask", model="baseten/docs-example-gated-model", use_auth_token=self._secrets["hf_access_token"] )
Storing secrets on your remote
On your remote host, such as your Baseten account, store both the secret name and value before deploying your model. On Baseten, you can add secrets to your workspace on the secrets workspace settings page.
Make sure to use the same name (case sensitive) as used in the Truss on the remote.
Deploying with secrets
For additional security, models don’t have access to secrets by default. To deploy a model and give it access to secrets, pass the
--trusted flag during
truss push as follows:
truss push --trusted
Your model will be deployed with access to secrets stored on your remote.