How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin

PUBLISHER ARTICLE PREVIEW

From the original article

Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize output within the factory’s limited power budget. This makes performance per watt—rather than raw, unnormalized throughput—the ultimate measure of an AI platform’s value.

The NVIDIA Vera Rubin platform is designed to enable power-efficient AI at scale. At its core is NVIDIA Vera Rubin NVL72, which delivers strong performance per watt across the widest range of AI compute demands—from throughput-optimized large batches to the small batches of high-interactivity tiers, for both open and closed models.

Factory and rack-level power management innovations drive Vera Rubin performance on this important metric. At the factory level, NVIDIA DSX MaxLPS software shifts power between racks as workloads demand shifts, allowing an operator to recover stranded power and provision up to 40% more GPUs within the same site-power envelope and deliver 35% higher token throughput.

Inside each Vera Rubin NVL72, rack-level capacitors, along with state-of-charge Intelligent Power Smoothing software, absorb the bursty power spikes of training and inference workloads, so the factory can be planned around sustained demand instead of worst-case peaks. This means more deployable compute per megawatt.

The highest interactivity tiers present unique challenges. For these, the platform adds NVIDIA Groq 3 LPX as a low-latency accelerator. This post explains innovations within individual LPX racks that make it a power-efficient contributor to the Vera Rubin platform, including its deterministic execution model.

The Groq 3 LPX deterministic execution model allows the LPU compiler to create a schedule of exactly when each piece of data will move to which specific compute unit and precisely when that operation will execute, down to the clock cycle. It extends to all 256 LPU chips in the rack, enabling ultrafast interactivity at long context and power management techniques that take advantage of this determinism.

After the LPU compiler has generated the workload execution schedule, it can predict the electrical current draw for each cycle in that schedule. This in turn enables two complementary technologies:

Together, PEP and CPS help reduce the “voltage guardband” or “electrical safety margin” that must be continually provided to any set of chips, despite not directly powering the AI workload. Decreasing this voltage guardband means a greater proportion of scarce power can be spent on the workload.

To run AI workloads,