NVIDIA’s Blackwell Ultra GB300 has delivered a major performance jump in a new agentic AI benchmark, showing up to 20x better performance per megawatt than the older H200 Hopper platform. The result highlights how quickly AI hardware is moving from basic model serving toward large scale agent workloads, where many AI agents must run at the same time while handling reasoning, coding tasks, tool calls, and changing context sizes.
The benchmark comes from Artificial Analysis through a new test called AA AgentPerf. Unlike simple token speed tests, this benchmark focuses on how many active AI agents an inference setup can support under realistic workloads. That makes it especially relevant as companies build AI systems that do more than answer one prompt at a time.
NVIDIA tested the GB300 NVL72 platform using DeepSeek V4 Pro, a frontier style model designed for advanced AI agent use cases. The results show Blackwell Ultra far ahead of Hopper in both power efficiency and per GPU serving capacity.
AA AgentPerf measures how AI systems handle real agent workloads
Agentic AI workloads are more complex than simple chatbot requests. They often involve multi turn reasoning, tool use, coding sessions, variable context lengths, and continuous concurrent requests. That means the hardware must manage not only raw compute but also memory, scheduling, KV cache reuse, and latency.
AA AgentPerf focuses on three key measurements.
| Metric | What it measures |
|---|---|
| Time to First Token | How long it takes before the first output token appears |
| Output Speed | How many output tokens per second are generated after the first token |
| System Output Throughput | Total output tokens per second across all active agents |
These metrics matter because AI agents are expected to work continuously rather than respond once and stop. In coding, research, automation, and enterprise workflows, many agents may be running at the same time.
GB300 NVL72 posts a major efficiency lead over H200
The headline number is the 20x performance improvement per megawatt over NVIDIA’s older H200 platform. According to the benchmark figures, GB300 NVL72 can support about 61.4K concurrent agents per megawatt. The H200 platform reaches about 2.6K concurrent agents per megawatt.
| Benchmark result | NVIDIA GB300 NVL72 | NVIDIA H200 |
|---|---|---|
| Concurrent agents per megawatt | 61.4K | 2.6K |
| Concurrent agents per GPU | 57.5 | 1.4 |
| Main advantage | Higher efficiency and serving capacity | Older Hopper generation baseline |
The per GPU number is also important. GB300 reportedly supports 57.5 concurrent agents per GPU, compared with just 1.4 on H200. That shows how much NVIDIA has improved serving density for agent workloads.
Blackwell Ultra is built for high concurrency AI deployments
The results suggest that GB300 is not only faster in raw compute terms but also much better suited for keeping GPUs busy across many concurrent sessions. That is essential for AI infrastructure providers, because idle hardware wastes money and power.
Agentic AI workloads can be unpredictable. One agent may be generating code, another may be calling tools, another may be reasoning through a task, and another may be waiting on context. Hardware and software must keep all of this moving efficiently.
GB300’s advantage appears to come from a combination of Blackwell architecture improvements, NVL72 scale, better memory handling, and NVIDIA’s inference software stack.
Rubin could push the gap even further
NVIDIA is already looking beyond Blackwell Ultra. Its upcoming Rubin architecture is expected to build on these gains with higher compute performance and tighter integration with the Vera CPU.
Rubin is expected to offer 50 PFLOPs of NVFP4 compute, which could make it even stronger for large language model inference and agentic workloads. The Vera CPU may also help improve tool calls and end to end AI pipeline performance, which matters more as AI systems become less like single models and more like connected agent platforms.
If Rubin lands on schedule and delivers the expected efficiency gains, NVIDIA could further extend its lead in the AI accelerator market.
Agentic AI is becoming the next big hardware test
The GB300 benchmark result shows why traditional AI benchmarks are no longer enough. The market is moving toward AI systems that can plan, act, call tools, write code, test results, and continue working across multiple steps. That requires different measurements than simple prompt completion.
For cloud providers and enterprises, the key question is no longer only how fast a model can respond. It is how many useful agents can run within a fixed power and cost budget.
That is why concurrent agents per megawatt matters. Data centers are limited by power, cooling, and rack density. A 20x efficiency gain can translate into more AI capacity without needing the same level of infrastructure expansion.
NVIDIA is strengthening its AI infrastructure lead
NVIDIA already dominates AI training and inference, and GB300’s performance in AA AgentPerf gives it another strong argument for agentic AI deployments. The company can now point to large gains not only in raw model performance but also in practical serving capacity for modern AI workloads.
The main challenge for competitors is that the goalposts keep moving. AMD, Intel, and custom AI chip makers are not only chasing Hopper or H200 anymore. They now have to compete against Blackwell Ultra, while Rubin is already approaching.
For AI companies building large scale agent systems, GB300 looks like a major step forward. It reduces the cost and power burden of running many agents at once, which may become one of the most important performance measures in the next phase of AI infrastructure.
Discussion (0)
Be the first to comment.