Deploying Inference Graphs to Kubernetes (dynamo deploy) — Dynamo
AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.
Title: Deploying Inference Graphs to Kubernetes (dynamo deploy)#
URL Source: https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/README.html?userAgent=PromptingBot%2F1.0.0
Published Time: Fri, 13 Jun 2025 20:22:51 GMT
Markdown Content: This guide explains the deployment options available for Dynamo inference graphs in Kubernetes environments.
Deployment Options#
Dynamo provides two distinct deployment options that each serve different use cases:
-
Dynamo Cloud Kubernetes Platform is preferred in cases that support it
-
Manual Deployment with Helm Charts is suited to users who need more control over their deployments
Dynamo Cloud Kubernetes Platform [PREFERRED]#
The Dynamo Cloud Platform (deploy/cloud/) provides a managed deployment experience:
-
Contains the infrastructure components required for the Dynamo cloud platform
-
Used when deploying with the
dynamo deployCLI commands -
Provides a managed deployment experience
For detailed instructions on using the Dynamo Cloud Platform, see:
-
Dynamo Cloud Platform Guide: walks through installing and configuring the Dynamo cloud components on your Kubernetes cluster.
Manual Deployment with Helm Charts#
Users who need more control over their deployments can use the manual deployment path (deploy/helm/):
-
Used for manually deploying inference graphs to Kubernetes
-
Contains Helm charts and configurations for deploying individual inference pipelines
-
Provides full control over deployment parameters
-
Requires manual management of infrastructure components
-
Documentation:
-
Using the Deployment Script: all-in-one script for manual deployment
-
Helm Deployment Guide: detailed instructions for manual deployment
-
Getting Started with Helm Deploy#
- For Dynamo Cloud Platform:
* Follow the [Dynamo Cloud Platform Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/dynamo_cloud.html.md)
* Deploy a Hello World pipeline using the [Operator Deployment Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/operator_deployment.html.md)
* Deploy a Dynamo LLM pipeline to Kubernetes [Deploy LLM Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/examples/llm_deployment.html.md#deploy-to-kubernetes)
* Model caching with [Fluid](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/model_caching_with_fluid.html.md)
2. For Manual Deployment:
* Follow the [Manual Helm Deployment Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/manual_helm_deployment.html.md)
Example Deployments#
See the Hello World example for a complete walkthrough of deploying a simple inference graph.
See the LLM example for a complete walkthrough of deploying a production-ready LLM inference pipeline to Kubernetes.
Links/Buttons:
- Skip to main content
- document.write(
<img src="../../_static/nvidia-logo-horiz-rgb-wht-for-screen.svg" class="logo__image only-dark" alt="Dynamo - Home"/>); Dynamo - GitHub
- Support Matrix
- Getting Started
- High Level Architecture
- Distributed Runtime
- Disaggregated Serving
- KV Block Manager
- Motivation
- KVBM Architecture
- Understanding KVBM components
- KVBM Further Reading
- KV Cache Routing
- Planner
- CLI Overview
- Running Dynamo (dynamo run)
- Serving Inference Graphs (dynamo serve)
- Building Dynamo (dynamo build)
- Deploying Inference Graphs (dynamo deploy)
- Writing Python Workers in Dynamo
- Disaggregation and Performance Tuning
- KV Cache Router Performance Tuning
- Planner Benchmark Example
- Working with Dynamo Kubernetes Operator
- Dynamo Cloud Kubernetes Platform
- Deploying Dynamo Inference Graphs to Kubernetes using the Dynamo Cloud Platform
- Manual Helm Deployment
- Minikube Setup Guide
- Model Caching with Fluid
- Dynamo SDK
- Python API
- Hello World Example: Basic
- Hello World Example: Aggregated and Disaggregated Deployment
- LLM Deployment Examples
- Multinode Examples
- LLM Deployment Examples using TensorRT-LLM
- #
- Using the Deployment Script
- Helm Deployment Guide
- Deploy LLM Guide
- Hello World example
- Privacy Policy
- Manage My Privacy
- Do Not Sell or Share My Data
- Terms of Service
- Accessibility
- Corporate Policies
- Product Security
- Contact