nvidia.com

Command Palette

Search for a command to run...

Last updated: 12/12/2025

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

Title: Dynamo Runtime — NVIDIA Dynamo Documentation

URL Source: https://docs.nvidia.com/dynamo/archive/0.6.1/development/runtime-guide.html

Published Time: Sat, 08 Nov 2025 00:28:59 GMT

Markdown Content: Skip to main content

Back to top Ctrl+K

Image 2: NVIDIA Dynamo Documentation - HomeImage 3: NVIDIA Dynamo Documentation - Home NVIDIA Dynamo Documentation

latest

latest0.6.10.6.00.5.10.5.00.4.10.4.00.3.20.3.10.3.00.2.10.2.0

Search Ctrl+K

Search Ctrl+K

Image 4: NVIDIA Dynamo Documentation - HomeImage 5: NVIDIA Dynamo Documentation - Home NVIDIA Dynamo Documentation

latest

latest0.6.10.6.00.5.10.5.00.4.10.4.00.3.20.3.10.3.00.2.10.2.0

Table of Contents

Getting Started

Kubernetes Deployment

User Guides

Components

Design Docs

Dynamo Runtime#

A Datacenter Scale Distributed Inference Serving Framework

Image 6: License

Rust implementation of the Dynamo runtime system, enabling distributed computing capabilities for machine learning workloads.

Prerequisites#

Install Rust and Cargo using rustup:#

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Build#

cargo build cargo test

Start Dependencies#

Docker Compose#

The simplest way to deploy the pre-requisite services is using docker-compose, defined in deploy/docker-compose.yml.

At the root of the repository:

docker compose -f deploy/docker-compose.yml up -d

This will deploy a NATS.io server and an etcd server used to communicate between and discover components at runtime.

Local (alternate)#

To deploy the pre-requisite services locally instead of using docker-compose above, you can manually launch each:

Run Examples#

When developing or running examples, any process or user that shared your core-services (etcd and nats.io) will be operating within your distributed runtime.

The current examples use a hard-coded namespace. We will address the namespace collisions later.

All examples require the etcd and nats.io pre-requisites to be running and available.

Rust hello_world#

With two terminals open, in one window:

cd examples/hello_world cargo run --bin server

In the second terminal, execute:

cd examples/hello_world cargo run --bin client

which should yield some output similar to:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.25s Running `target/debug/client`

Annotated { data: Some("h"), id: None, event: None, comment: None } Annotated { data: Some("e"), id: None, event: None, comment: None } Annotated { data: Some("l"), id: None, event: None, comment: None } Annotated { data: Some("l"), id: None, event: None, comment: None } Annotated { data: Some("o"), id: None, event: None, comment: None } Annotated { data: Some(" "), id: None, event: None, comment: None } Annotated { data: Some("w"), id: None, event: None, comment: None } Annotated { data: Some("o"), id: None, event: None, comment: None } Annotated { data: Some("r"), id: None, event: None, comment: None } Annotated { data: Some("l"), id: None, event: None, comment: None } Annotated { data: Some("d"), id: None, event: None, comment: None }

Python#

See the README.md for details

The Python and Rust hello_world client and server examples are interchangeable, so you can start the Python server.py and talk to it from the Rust client.

On this page

Image 7: NVIDIAImage 8: NVIDIA

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2024-2025, NVIDIA CORPORATION & AFFILIATES.

Image 9Image 10Image 11

Links/Buttons:

Related Articles