Used Optane memory helps run a 1 trillion parameter AI model on a single GPU workstation

A Reddit user has shown that old Intel Optane Persistent Memory can still be useful for modern AI workloads, even after Intel killed the product line. The user built a Xeon workstation with 768GB of second hand Optane PMem DIMMs and managed to run Kimi K2.5, a 1 trillion parameter language model, locally at around 4 tokens per second.

The result is not fast compared with expensive AI servers, but that is not really the point. Running a model of that size on a single GPU workstation is unusual because frontier scale models normally need huge memory pools, multiple accelerators, and costly data center hardware. This build shows that cheap used Optane can fill an unusual gap between normal DRAM and NVMe SSD storage.

Optane is slower than DRAM, but it can still be useful when capacity matters more than speed

The workstation used six 128GB Intel Optane DCPMM modules for a total of 768GB of persistent memory. These Optane DIMMs were configured in memory mode, while six 32GB Samsung DDR4 ECC sticks acted as cache. The system also used an Intel Xeon Gold 6246 CPU, a Tyan motherboard, an Asus Dual GeForce RTX 3060 OC with 12GB of VRAM, and a 2TB WD SN850X NVMe SSD.

Optane PMem was originally designed to sit between DRAM and SSDs. It has much lower latency than even fast NVMe storage, but it is still slower than DRAM. That makes it awkward for many normal workloads, but interesting for AI inference, where a large model may need huge memory capacity more than raw memory speed.

The software setup used llama.cpp with a hybrid GPU and CPU inference approach. The user also placed the model’s routing components into the 12GB GPU by using llama.cpp’s override tensor option. That helped make better use of the limited GPU memory while the larger memory pool handled the rest of the model.

The performance result, around 4 tokens per second, will not impress anyone used to high end AI servers. But for a system built from used hardware and only one consumer class GPU, it is still notable. A 1 trillion parameter model is far beyond what most home AI setups can load normally.

The bigger lesson is that AI workloads are exposing a memory gap. DRAM is fast but expensive. SSDs are cheap and large but too slow for many active inference tasks. Optane once tried to fill that middle space, but Intel ended the product line before AI workloads became as mainstream as they are now.

That timing is why this build is interesting. Used Optane modules are now cheaper than equivalent DRAM capacity, and they can give old Xeon workstations unusually large memory pools. For hobbyists, researchers, and local AI experimenters, that creates a small second life for a technology that failed commercially but still has technical value.

The downside is that this is an exotic setup. Optane PMem needs compatible platforms, older server hardware, and careful configuration. It is not something normal PC builders can add to a gaming desktop. Intel has also discontinued Optane, so the supply depends on second hand markets.

The project also points toward where future systems may go. Technologies such as CXL could eventually provide large, affordable, byte addressable memory pools for AI and other memory heavy workloads. In a way, this Optane experiment shows the kind of role that future memory expansion standards may need to fill.

For now, the Redditor’s build is a clever workaround rather than a mainstream path. It proves that old Optane DIMMs can still do something impressive: help a single GPU workstation load and run a massive local AI model that would normally be out of reach for typical home hardware.