Making Kubernetes AI-Ready for NVIDIA Vera Rubin on Day One

PUBLISHER ARTICLE PREVIEW

From the original article

Configuring and optimizing Kubernetes for new GPU hardware such as NVIDIA Vera Rubin is complex and error-prone. Drivers, kernels, runtimes, networking, operators, and schedulers must work together, with settings tuned to the hardware and workload. Small mismatches can prevent workloads from running or limit performance.

The NVIDIA AI Cluster Runtime (AICR)—part of the NVIDIA DSX AI Factory Platform—simplifies this work by capturing tuned configurations in reusable recipes, generating bundles for existing deployment tools, and validating the running cluster. It will help Kubernetes environments be ready for NVIDIA Vera Rubin NVL72 compute trays on Day 1.

Overall, the NVIDIA DSX AI Factory Platform brings together technologies to help the ecosystem design, build, and operate more efficient AI factories, including open source, modular software components purpose-built to support multi-tenant operations and scaling.

Configuring new hardware architecture in a cloud-native stack requires plumbing updates from the lowest-level bindings all the way up to the orchestration layer. It is still early days for Vera Rubin.

This post covers what changes if you run NVIDIA GB200 NVL72 today, what has been validated on early hardware access using NVIDIA GPU Operator and the Dynamic Resource Allocation (DRA) driver with Vera Rubin support. It also shows how AICR brings these components together in a configuration that platform teams can reproduce and verify.

If you run GB200 NVL72 today, NVIDIA Vera Rubin NVL72 is an evolution of the same NVIDIA MGX rack-scale architecture, now in its third generation. But several user-visible pieces change.

To validate Vera Rubin NVL72 support end to end, we deployed the AICR runtime bundle on Vera Rubin clusters using the official GPU Operator v26.7.0 release and the official DRA driver v0.5.0 release, with a host-installed R615 driver.

Both the training and inference recipe variants now validate end to end for deployment and conformance. For training, an initial performance gate: a cross-node NCCL all-reduce over the NVIDIA NVLink fabric, running through an IMEX ComputeDomain and gated on a bus-bandwidth floor. GPU pods run cleanly, and IMEX channels are injected into claiming pods across multiple nodes, shown with real cluster output below.

A note on early-silicon realities: Bringing up new silicon is a process. The first validation rounds were temporarily blocked by a hardware/fabric issue in one of the early Vera Rubin clusters—unrelated to the GPU Operator or the DRA driver—that kept one compute tray out of the NVLink fabric. After the hardware was reseated, all GPUs on both nodes joined a single clean fabric clique.

Validating the components is only half the story. Standing up a working Vera Rubin cluster means choosing driver mode, container-runtime wiring, component versions, and a handful of platform-specific settings. AICR captures those choices as a versioned recipe, so the configuration on real Vera Rubin hardware is explicit, reproducible, and verifiable across clusters.