NVIDIA has detailed the architecture behind its next generation Rubin GPU, revealing a 336 billion transistor design built to deliver much higher performance for agentic AI, mixture of experts models, and large scale inference.
Rubin is manufactured using TSMC’s N3P process and combines two large dies inside one package. NVIDIA connects the dies through its high bandwidth NV HBI interface so the complete accelerator can operate as a unified GPU.
The transistor count is about 62 percent higher than the Blackwell GB300. NVIDIA says the architectural changes allow Rubin to provide up to ten times more agentic throughput per unit of energy than Blackwell when deployed with the Vera CPU inside an NVL72 system.
The gains come from three main areas: upgraded Tensor Cores, a new HBM4 memory subsystem, and the third generation Transformer Engine.
Rubin combines 224 Streaming Multiprocessors with 896 Tensor Cores
Each Rubin GPU includes eight Graphics Processing Clusters, with four located on each die. Across the full package, the chip contains 224 Streaming Multiprocessors and 896 Tensor Cores.
The internal layout does not appear completely uniform. The block diagram indicates that some Graphics Processing Clusters contain 30 Streaming Multiprocessors, while others have 26.
Each die also includes four HBM channels, large decentralised L2 cache blocks, a Gigathread Engine, and an NVLink interface.
| Rubin specification | Confirmed detail |
|---|---|
| Manufacturing process | TSMC N3P |
| Transistor count | 336 billion |
| GPU dies | Two |
| Graphics Processing Clusters | Eight |
| Streaming Multiprocessors | 224 |
| Tensor Cores | 896 |
| HBM4 capacity | 288GB |
| HBM4 bandwidth | Up to 22TB/s |
| NVLink 6 bandwidth | 3.6TB/s |
| CPU to GPU bandwidth | 1.8TB/s |
| Host interface | PCIe 6.0 x16 |
The GPU also supports Multi Instance GPU partitioning, allowing one accelerator to be divided between several workloads. A dedicated decoding block handles accelerated video processing, while confidential computing features protect information while it is stored, transferred, and processed.
HBM4 increases bandwidth to 22TB per second
Rubin uses 288GB of HBM4 memory distributed across eight 12 layer stacks. Each stack provides 36GB of capacity.
The memory subsystem reaches a peak bandwidth of 22TB/s. That is 2.8 times higher than the 8TB/s available on the compared Blackwell and Blackwell Ultra platforms.
The additional bandwidth is important for inference because token generation frequently depends on moving model weights and key value cache data rather than raw arithmetic performance alone.
| Memory platform | Capacity | Bandwidth |
|---|---|---|
| Rubin HBM4 | 288GB | 22TB/s |
| Blackwell HBM3e | Varies by system | 8TB/s |
| Blackwell Ultra HBM3e | Varies by system | 8TB/s |
Higher capacity allows larger models, longer context windows, and more simultaneous requests to remain in memory. It can also reduce the need to move key value caches to slower storage or another accelerator.
The upgraded Tensor Memory Accelerator helps software move complicated data layouts through the GPU more efficiently. NVIDIA has also added memory locality improvements intended to keep frequently used information closer to the processing resources that need it.
Enhanced Tensor Cores process more data per clock
Rubin’s Tensor Cores can process twice as much data along the K dimension during each clock cycle.
A matrix operation that required four K iterations on Blackwell may need only two iterations on Rubin. Reducing the number of loops can improve both compute heavy operations and workloads limited by latency or memory movement.
The new Tensor Cores deliver up to 50 PFLOPS of NVFP4 performance. Rubin also doubles FP32 throughput per clock and per Streaming Multiprocessor compared with the baseline GB200.
BF16 and FP16 throughput is four times higher than GB200 and twice as high as GB300 Blackwell Ultra.
The architecture also improves attention processing by combining activation sparsity with adaptive compression. Intermediate attention data can be stored in a structured 2:4 sparse format, reducing the amount of information that later stages need to process.
This lowers storage and movement requirements without changing the dense output expected by the rest of the model.
Mixture of experts models receive more efficient data handling
Modern mixture of experts models may contain a large number of specialised expert networks. Only some experts are used for each request, but the system must still locate and move the correct weights quickly.
Rubin’s improved Tensor Memory Accelerator can share one descriptor across several experts that use the same tensor layout. Software can then change values such as memory pointers and strides at runtime.
Blackwell may require a separate descriptor for each expert. Rubin’s approach reduces management overhead and gives the accelerator more time for useful inference work.
This could become increasingly important as models add more experts and route more requests dynamically.
Fine grained kernel overlap reduces idle time
Rubin introduces more precise coordination between connected kernels.
A consumer kernel can begin operating as soon as the required input becomes available rather than waiting for a larger producer workload to finish completely.
This creates a tighter execution timeline with fewer idle gaps. It is particularly useful for agentic inference, where information moves through several stages and small delays between kernels can directly affect token generation speed.
The architecture also improves GPU communication through NVLink 6. Each Rubin GPU provides 3.6TB/s of bandwidth to the NVLink switch.
The coherent NVLink C2C interface offers 1.8TB/s between the CPU and GPU, while PCIe 6.0 x16 provides up to 256GB/s for communication with a host processor.
Rubin systems reduce power spikes and support denser deployments
NVIDIA has also changed how Vera Rubin systems manage power.
The platform uses intelligent power smoothing supplies with local energy storage to absorb short term demand spikes. NVIDIA says this reduces average power use by approximately 10 percent compared with the previous generation.
Peak power over a 50 millisecond period can fall by around 20 percent. Reducing short spikes allows data centres to use their available electrical capacity more effectively.
NVIDIA’s DSX MaxLPS software manages power across GPUs, racks, workloads, and liquid cooling systems. The company claims it can allow up to 40 percent more GPUs to operate within the same power budget.
These gains will depend on facility design, workload behaviour, and rack configuration. However, power management has become increasingly important because electricity availability now limits the size of many AI deployments.
Rubin forms the core of NVIDIA’s next AI platform
Rubin is designed to operate alongside NVIDIA’s Vera CPU inside rack scale systems rather than as an isolated accelerator.
The combination is intended to provide up to ten times more agentic workload throughput per unit of energy and twice as many tool calls compared with a Blackwell NVL72 system using traditional x86 processors.
The architecture focuses heavily on model memory, token generation, communication, and power efficiency. These areas have become just as important as peak arithmetic performance for large AI deployments.
Independent testing will be needed to confirm the full performance claims across different models and software frameworks. Still, the disclosed specifications show that Rubin is a substantial architectural change rather than a simple increase in core count.
Its 336 billion transistors, 22TB/s HBM4 subsystem, upgraded Tensor Cores, and faster interconnects give NVIDIA a platform designed for increasingly large models and more complex agent based workloads.
Discussion (0)
Be the first to comment.