Getting started
Welcome to Truss
The simplest way to serve AI/ML models in production
Why Truss?
- Write once, run anywhere: Package and test model code, weights, and dependencies with a model server that behaves the same in development and production.
- Fast developer loop: Implement your model with fast feedback from a live reload server, and skip Docker and Kubernetes configuration with a batteries-included model serving environment.
- Support for all Python frameworks: From
transformers
anddiffusers
toPyTorch
andTensorflow
toXGBoost
andsklearn
, Truss supports models created with any framework, even entirely custom models.
See Trusses for popular models including:
- 🦙 Llama 2 7B (13B) (70B)
- 🎨 Stable Diffusion XL
- 🗣 Whisper
and dozens more examples on GitHub.
Deploy your first model
Quickstart
Package, deploy, and invoke an ML model in production all in less than five minutes.
Truss tutorial
Learn model deployment step-by-step from “Hello, World!” to streaming output from an open-source LLM.
Truss contributors
Truss is backed by Baseten and built in collaboration with ML engineers worldwide. Special thanks to Stephan Auerhahn @ stability.ai and Daniel Sarfati @ Salad Technologies for their contributions.
We enthusiastically welcome contributions in accordance with our contributors’ guide and code of conduct.