NVIDIA's Tesla V100 may be nearly nine years old, but a custom four GPU setup shows that the Volta based accelerator can still deliver competitive performance in modern AI inference workloads.
The experiment used four 32 GB Tesla V100 cards connected through NVLink and compared them with two DGX Spark systems and an Apple M5 Ultra. Despite the age of the hardware, the V100 cluster recorded substantial advantages in several Qwen3.8 Flash Next benchmarks, including a result that was 230% faster than the M5 Ultra in long prompt processing.
The setup required considerable modification, so it is not a simple alternative to buying a modern AI workstation.
Four V100 GPUs Required a Custom NVLink Setup
The Tesla V100 launched with NVIDIA's Volta architecture in 2017. The model used in the test carries 32 GB of HBM2 memory and offers up to 900 GB/s of memory bandwidth.
Running four of the GPUs together required an NVLink carrier board.
The board provided six NVLink 2.0 connections and around 300 GB/s of interconnect bandwidth. It was connected to the host system through a PCIe x8 link using passthrough adapters.
Cooling also needed modification.
Standard cooling was not suitable for the unusual arrangement, so NVIDIA A100 heatsinks were physically milled down to fit the V100 GPUs. Each modified cooler was then able to handle roughly 300W of heat from its GPU.
| Hardware | Key detail |
|---|---|
| Tesla V100 | 32 GB HBM2 per GPU |
| Memory bandwidth | Up to 900 GB/s |
| Number of GPUs tested | Four |
| NVLink generation | NVLink 2.0 |
| Interconnect bandwidth | Around 300 GB/s |
| Approximate GPU power | 300W each |
| Model tested | Qwen3.8 Flash Next |
| Quantization | 4 bit |
The physical complexity makes the setup very different from compact systems such as DGX Spark or Apple Silicon machines.
V100 Cluster Beats DGX Spark in Prompt Processing
The four V100 GPUs were tested against a pair of DGX Spark systems carrying 128 GB of LPDDR5X unified memory.
With Qwen3.8 Flash Next running at 4 bit quantization, the older NVIDIA accelerators performed particularly well in prompt processing.
For long prompts, the four V100 cards were reportedly around 40% faster than the dual DGX Spark configuration.
Against the M5 Ultra, the difference was much larger, with the V100 cluster reaching roughly 230% higher prompt processing performance.
The test author noted that the large gap against Apple Silicon may partly reflect software optimization differences rather than hardware capability alone.
That distinction matters because AI benchmark results can change substantially depending on framework support, kernels, quantization methods and how well software is optimized for a particular architecture.
Short Prompt Tests Show Similar Advantages
The four V100 setup maintained approximately the same 40% advantage over the two DGX Spark systems during shorter prompt processing.
Compared with the M5 Ultra, it was reported to be around 163% faster in that workload.
Token generation also favored the older GPUs.
The V100 configuration reached 100.6 tokens per second in the short prompt token generation test. That was described as 121% faster than the pair of DGX Spark systems.
The M5 Ultra achieved 44.5 tokens per second in a separately reported comparison.
Another test showed the quad V100 system reaching 155.2 tokens per second in token generation and 4,098.3 tokens per second for prompt processing in Qwen3.8 Flash Next.
Results from a separate eight GPU RTX PRO 6000 system were also included for comparison, although that much newer and substantially more expensive configuration remained in a different performance class.
Old Data Center GPUs Still Have Useful AI Hardware
The results demonstrate why older enterprise accelerators continue to attract interest among people building local AI systems.
The Tesla V100 was designed long before today's generative AI boom, but it already included hardware aimed at machine learning workloads along with high bandwidth HBM2 memory.
That combination can still be useful for running large language models, particularly when multiple cards can be connected through NVLink.
The V100 has also appeared in other unusual home AI builds. One earlier experiment paired a Tesla V100 with an RTX 4080 in the same system, using the older accelerator's additional memory and bandwidth to help run 27 billion parameter models.
Used Pricing Makes the V100 Interesting but Not Simple
Used 32 GB Tesla V100 cards were listed for around $744 in the market referenced by the report.
On paper, that can make two or more cards attractive compared with some modern AI systems, especially for workloads that benefit from large amounts of GPU memory.
However, the purchase price is only part of the equation.
A multi V100 system requires significant power, cooling, physical space and compatible interconnect hardware. Four cards running at around 300W each can consume roughly 1,200W before accounting for the rest of the system.
Modern alternatives can therefore offer major advantages in efficiency, size, software support and ease of deployment even when the older hardware wins individual benchmarks.
Still, the experiment shows that Volta has not become irrelevant. With the right software and enough engineering effort, four Tesla V100 GPUs can still provide serious local inference performance, even against much newer AI hardware.
Discussion (0)
Be the first to comment.