Nvidia’s new Vera Rubin platform debuted at Siggraph 2026. It marks a turn in AI infrastructure from model training toward sustained agentic inference. The chip targets long context, mixture-of-experts routing, decode throughput, memory locality, and rack-level power control. For ISVs, silicon teams, and CIOs, the message comes through clearly: the next fight does not stop at tensor cores. It runs through memory bandwidth, synchronization, interconnect, cooling, and software that can keep an AI factory producing useful tokens inside a fixed power envelope at scale.
Agentic AI changes the job description for the GPU. An agent has to reason, plan, call tools, check intermediate results, retrieve information, revise its path, and keep moving across long context. That workload stresses decode, attention, memory bandwidth, synchronization, and scale-up communication. It also turns energy efficiency into a system problem.
That context explains Vera Rubin. Nvidia designed the Rubin GPU for a world where inference has become a continuous production workload. The company says the Vera Rubin platform can deliver up to 10x more agentic throughput per unit of energy than Blackwell on an internal 2T mixture-of-experts workload. The claim works as a statement of direction. Nvidia wants the industry to measure AI infrastructure by useful agent output per watt as the primary figure.
Rubin starts with compute density. The GPU uses two reticle-limited compute dies joined through Nvidia’s high-bandwidth interface. Nvidia lists 336 billion transistors, 224 streaming multiprocessors, and 896 tensor cores. The third-generation transformer engine supports flexible precision and delivers up to 50 petaflops of sparse NVFP4 inference performance.
Memory carries much of that burden. Rubin supports up to 288 GB of HBM4 per GPU and up to 22 TB/s of peak memory bandwidth. HBM4 doubles the interface width relative to HBM3e, and Nvidia pairs it with a new memory controller and tighter compute-memory integration. Capacity helps with model residency, long context, KV (key value) cache, and concurrency. Bandwidth helps decode, where token generation pulls weights and KV state through the memory system.
That distinction matters for CIOs. A larger model needs local memory, bandwidth, and interconnect in balance. A procurement team that only compares peak math rates will miss the constraint that determines tokens per second per user.
Rubin also attacks the inherent mixture-of-experts overhead. MoE models route tokens through selected expert networks, the GPU locates and moves expert weights efficiently. Rubin enhances the tensor memory accelerator with inline descriptor update support. Software can keep one descriptor for tensors that share a layout and override fields such as pointer and stride at runtime. That reduces metadata work as expert count rises.
Matrix throughput changes too. Rubin doubles tensor core throughput per clock by processing twice as much data along the K dimension. A GEMM (General Matrix-to-Matrix Multiplication) that takes four K-loop iterations on Blackwell can take two on Rubin. That helps context and decode GEMMs at high tensor-parallel scale and shows up in overhead, utilization, and latency.
This table explains why the softmax machine learning function received attention in the design. As tensor core throughput rises, exponential math and reductions can become the limiting path. Rubin doubles FP32 exponential throughput and quadruples BF16/FP16 throughput versus Blackwell. That keeps attention from falling behind faster matrix operations.
Kernel launch and dependency handling get similar treatment. In inference, activations sit on the critical path. Rubin adds more fine-grained producer-consumer coordination so dependent work can begin as soon as required input data arrives. That reduces idle gaps between kernels and improves tokens per second per user.
Scale-up communication also moves closer to the compute path. Distributed inference requires frequent GPU-to-GPU traffic, reductions, and synchronization. Rubin introduces counted writes for device-initiated NVLink communication. A receiving GPU can track transfer completion with lower synchronization overhead, and fused kernels can keep work moving across NVLink without CPU coordination.
The interconnect numbers show the rack intent. Rubin provides NVLink 6 bandwidth of 3,600 GB/s to the NVLink switch for GPU-to-GPU scale-up communication. NVLink-C2C delivers 1,800 GB/s for coherent CPU-GPU communication, and x16 PCIe Gen 6 supplies up to 256 GB/s of host connectivity. These figures put Rubin inside a rack-scale execution model.
Power completes the story. Agentic inference produces bursty demand, and rack operators cannot size every part of the facility around short peaks. Vera Rubin NVL72 combines compute, networking, liquid cooling, rack-scale power steering, and intelligent power smoothing with energy storage. Nvidia says this approach can reduce average power consumption by about 10% and reduce 50 ms peak power by about 20%.
DSX MaxLPS extends that idea across racks, GPUs, 45 C liquid cooling, and workloads. Nvidia says operators can provision up to 40% more GPUs within the same power budget at energy-efficient operating points with limited workload impact. That message will get attention from CIOs, because data center power has become a gating factor. If software can recover stranded power, the system can add useful compute without waiting for a new substation.
Reliability also enters the buying decision. Vera Rubin NVL72 uses a third-generation MGX rack architecture with cable-free compute and switch trays, hot-swappable NVLink switch trays, 45 C liquid cooling, dynamic rack-scale power steering, and RAS improvements. The rack, management software, power controls, cooling path, and service model define the platform.
Mixture of experts– is an AI model design that routes each token to selected specialist sub-networks, or “experts,” instead of running every token through the whole model. For ISVs, Rubin creates a new optimization target. Software that understands MoE routing, memory descriptors, sparse activation paths, softmax pressure, and kernel dependency timing can extract more from the platform. For silicon teams, Rubin points to tensor math, HBM capacity, memory locality, synchronization, and the control paths that keep inference graphs moving. For CIOs, Rubin reframes procurement around work completed per watt, per rack, and per user interaction.
The industry has spent years talking about AI accelerators as chips. Rubin pushes the conversation toward infrastructure. The useful unit now looks more like a rack, pod, or factory slice than a single accelerator. That shift will influence model deployment, software optimization, power planning, and vendor selection. The next generation of AI systems will win by moving less data, waiting less often, wasting less power, and finishing more agent work inside the same facility envelope.
This table explains why the softmax machine learning function received attention in the design. As tensor core throughput rises, exponential math and reductions can become the limiting path. Rubin doubles FP32 exponential throughput and quadruples BF16/FP16 throughput versus Blackwell. That keeps attention from falling behind faster matrix operations.
Kernel launch and dependency handling get similar treatment. In inference, activations sit on the critical path. Rubin adds more fine-grained producer-consumer coordination so dependent work can begin as soon as required input data arrives. That reduces idle gaps between kernels and improves tokens per second per user.
Scale-up communication also moves closer to the compute path. Distributed inference requires frequent GPU-to-GPU traffic, reductions, and synchronization. Rubin introduces counted writes for device-initiated NVLink communication. A receiving GPU can track transfer completion with lower synchronization overhead, and fused kernels can keep work moving across NVLink without CPU coordination.
The interconnect numbers show the rack intent. Rubin provides NVLink 6 bandwidth of 3,600 GB/s to the NVLink switch for GPU-to-GPU scale-up communication. NVLink-C2C delivers 1,800 GB/s for coherent CPU-GPU communication, and x16 PCIe Gen 6 supplies up to 256 GB/s of host connectivity. These figures put Rubin inside a rack-scale execution model.
Power completes the story. Agentic inference produces bursty demand, and rack operators cannot size every part of the facility around short peaks. Vera Rubin NVL72 combines compute, networking, liquid cooling, rack-scale power steering, and intelligent power smoothing with energy storage. Nvidia says this approach can reduce average power consumption by about 10% and reduce 50 ms peak power by about 20%.
DSX MaxLPS extends that idea across racks, GPUs, 45 C liquid cooling, and workloads. Nvidia says operators can provision up to 40% more GPUs within the same power budget at energy-efficient operating points with limited workload impact. That message will get attention from CIOs, because data center power has become a gating factor. If software can recover stranded power, the system can add useful compute without waiting for a new substation.
Reliability also enters the buying decision. Vera Rubin NVL72 uses a third-generation MGX rack architecture with cable-free compute and switch trays, hot-swappable NVLink switch trays, 45 C liquid cooling, dynamic rack-scale power steering, and RAS improvements. The rack, management software, power controls, cooling path, and service model define the platform.
Mixture of experts– is an AI model design that routes each token to selected specialist sub-networks, or “experts,” instead of running every token through the whole model. For ISVs, Rubin creates a new optimization target. Software that understands MoE routing, memory descriptors, sparse activation paths, softmax pressure, and kernel dependency timing can extract more from the platform. For silicon teams, Rubin points to tensor math, HBM capacity, memory locality, synchronization, and the control paths that keep inference graphs moving. For CIOs, Rubin reframes procurement around work completed per watt, per rack, and per user interaction.
The industry has spent years talking about AI accelerators as chips. Rubin pushes the conversation toward infrastructure. The useful unit now looks more like a rack, pod, or factory slice than a single accelerator. That shift will influence model deployment, software optimization, power planning, and vendor selection. The next generation of AI systems will win by moving less data, waiting less often, wasting less power, and finishing more agent work inside the same facility envelope.
Vera Rubin gives Nvidia a clear technical argument for agentic AI: keep data closer, move it with less overhead, coordinate kernels sooner, communicate across GPUs with less synchronization, and manage power at rack scale. That argument speaks to both sides of the market. Engineers get a deeper set of hardware hooks. Operators get a way to think about throughput inside fixed power, cooling, and space limits. Rubin turns a GPU discussion into a system design discussion.
What do we think?
Nvidia has moved the performance argument from peak GPU math to useful agentic throughput. Rubin’s value sits in memory bandwidth, HBM4 capacity, attention handling, NVLink coordination, and rack power control. Customers with software teams can exploit more of that stack. CIOs should treat simple accelerator comparisons as incomplete, especially for long-context, high-concurrency inference deployments.
Rubin represents an inflection point in AI infrastructure, because it treats inference as an always-on factory workload made from linked model calls. The important signal sits in the blend of HBM4, sparse attention, counted writes, and rack-scale power control. If agentic AI grows as expected, buyers will look at completed work per watt, per rack, and per latency target, with raw accelerator specs serving as only one input.
Nvidia’s Rubin is one of the 288 AI processors we are tracking in our AIP Tracker/information system, along with the 152 other AIP suppliers, with most them looking for ways to take Nvidia’s lunch money. Investors however, have closed their purses and we’re not seeing the flamboyant 100s of millions off dollars investments. No industry can sustain this many vendors. In addition to the AIP information service, we offer a quarterly update report.