Some models need access to APIs, databases, AWS resources, or other secured information. With Truss, you can securely reference access keys, API tokens, passwords, secrets, and more. Keep this sensitive information out of your Git repository and use the same best practices for ML models as production services.
This example is based on the GFP-GAN example Truss in the main repository. That model needs access to an S3 bucket on AWS, requiring an access key id, access key secret, AWS region, and bucket name. These values are handled as secrets.
Start by defining a list of secrets in the
config.yamlfile. YAML is a key-value store, but you don't want to ever actually store the secret values in this file, even when running the Truss locally. Instead, set default values for the secrets, like
null, and can leave comments about what kind of value to expect. Default values can also be fallback values that show format or type, like an empty string, the number 0, or an example UUID.
gfpgan_aws_region: null # e.g. us-east-1
Then, you can access the secrets in the
model/model.pyfile by referencing them as kwargs in the init function.
def __init__(self, **kwargs) -> None:
self._config = kwargs.get("config")
secrets = kwargs.get("secrets")
From there, you can use the secrets as a dictionary within the model file in any function.
# Still in __init__
self.s3_config = (
self.s3_bucket = (secrets["gfpgan_aws_bucket"])
When you run Truss locally, it creates a folder at
~/.trussto store configuration and temporary files. This folder may or may not contain a file
config.yaml. If that file doesn't exist, make it with:
Then add the same secret names that you use in your Truss config in this local config, but actually set the secret value in
When running your model in production (or a production-like environment such as staging), you have two options: environment variables and mounted secrets.
To set a secret value in production, you can set an environment variable and reference its value in your Truss. To avoid namespace conflicts, environment variable names must be prefixed with
TRUSS_SECRET_. For example:
Mounted secrets should not use the
TRUSS_SECRET_prefix as there is no need to avoid namespace conflicts.
If you're deploying your model to Baseten, set
deploy_trusscommand to enable your model to access secrets:
If you're deploying your model to another platform, reference that platform's documentation for setting environment variables or mounting a secrets volume.