nvidia.com

Command Palette

Search for a command to run...

Deploying Inference Graphs to Kubernetes (dynamo deploy) — Dynamo

Last updated: 12/12/2025

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

Title: Deploying Inference Graphs to Kubernetes (dynamo deploy)#

URL Source: https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/README.html?userAgent=PromptingBot%2F1.0.0

Published Time: Fri, 13 Jun 2025 20:22:51 GMT

Markdown Content: This guide explains the deployment options available for Dynamo inference graphs in Kubernetes environments.

Deployment Options#

Dynamo provides two distinct deployment options that each serve different use cases:

  1. Dynamo Cloud Kubernetes Platform is preferred in cases that support it

  2. Manual Deployment with Helm Charts is suited to users who need more control over their deployments

Dynamo Cloud Kubernetes Platform [PREFERRED]#

The Dynamo Cloud Platform (deploy/cloud/) provides a managed deployment experience:

  • Contains the infrastructure components required for the Dynamo cloud platform

  • Used when deploying with the dynamo deploy CLI commands

  • Provides a managed deployment experience

For detailed instructions on using the Dynamo Cloud Platform, see:

Manual Deployment with Helm Charts#

Users who need more control over their deployments can use the manual deployment path (deploy/helm/):

  • Used for manually deploying inference graphs to Kubernetes

  • Contains Helm charts and configurations for deploying individual inference pipelines

  • Provides full control over deployment parameters

  • Requires manual management of infrastructure components

  • Documentation:

Getting Started with Helm Deploy#

  1. For Dynamo Cloud Platform:
* Follow the [Dynamo Cloud Platform Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/dynamo_cloud.html.md) * Deploy a Hello World pipeline using the [Operator Deployment Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/operator_deployment.html.md) * Deploy a Dynamo LLM pipeline to Kubernetes [Deploy LLM Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/examples/llm_deployment.html.md#deploy-to-kubernetes) * Model caching with [Fluid](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/model_caching_with_fluid.html.md)

2. For Manual Deployment:

* Follow the [Manual Helm Deployment Guide](https://docs.nvidia.com/dynamo/archive/0.3.0/guides/dynamo_deploy/manual_helm_deployment.html.md)

Example Deployments#

See the Hello World example for a complete walkthrough of deploying a simple inference graph.

See the LLM example for a complete walkthrough of deploying a production-ready LLM inference pipeline to Kubernetes.

Links/Buttons:

Related Articles