Tensormesh just proved a genuinely different way to stretch GPU capacity: Don’t build bigger memory; stop throwing away the memory you already computed. The company’s KV cache software now works directly with AMD’s virtual memory tools, letting enterprises serve more AI models on the same GPUs instead of buying more of them. In testing, that combination cut response time nearly 7× and doubled how many models a server could run. We look at how the technology actually works, and why it matters right now.
Tensormesh doesn’t build chips or a stand-alone cache device. The company builds software, an engine called LMCache, that changes where a large language model’s KV cache lives and how long it survives. In a standard inference setup, the GPU’s own HBM holds the KV cache during a query, then discards it the moment the query finishes. Tensormesh keeps that cache alive instead, moving it out to CPU DRAM or network storage when the GPU needs the space back, then pulling it into GPU memory again when a similar query arrives. Nothing about the GPU hardware changes; the trick is entirely in the orchestration layer sitting on top of it.
Tensormesh announced a collaboration with AMD that pairs its KV cache software with AMD’s Live Context Virtualization, a virtual GPU memory tool. The two systems together let a server run more models on the same fixed set of GPUs and hold onto high cache hit rates and throughput, even when HBM capacity runs oversubscribed. Tensormesh tested the combination on Dell servers running eight AMD MI355 GPUs with Dell storage attached. LMCache coordinates the cache management across that stack, and customers get a direct benefit from infrastructure they already own: Existing GPU deployments effectively gain capacity without a hardware purchase, and cached KV chunks can get reused later for prefix or non-prefix matching across different queries.
The approach targets a real industry problem directly: Adding more memory to GPUs has been the default fix for capacity constraints, and that demand is a real contributor to today’s memory supply crunch. Tensormesh’s testing points at a cheaper path. On a 300 GB document set run against the Kimi-K2.6 model, time to first token dropped from 3.4 seconds to under half a second, close to a 7× improvement, by reusing cached KV data pulled from DRAM and network storage instead of recomputing it. Output throughput held near 48 token/sec, regardless of how much load hit the system, whereas an unoptimized setup lost close to 40% of its throughput under the same conditions. Model density on the identical hardware doubled.
(Source: Tensormesh)
This AMD collaboration builds on a direct financial relationship: AMD Ventures took part in a $20 million investment round in Tensormesh alongside CoreWeave and Nvidia’s venture arm, NVentures, disclosed in May 2026, on top of an earlier $4.5 million raised the previous October. Tensormesh CEO Junchen Jiang, who co-created LMCache, described the AMD work as extending the memory inference engines so they can draw on for both model weights and KV cache, pulling from every memory resource available on a node, not GPU memory alone. AMD’s Anush Elangovan, the company’s vice president of AI software, credited Tensormesh and LMCache as leaders in KV cache management and framed the joint work as a breakthrough in virtual GPU memory handling. Tensormesh has also built peer-to-peer CPU memory sharing, developed with Tencent’s infrastructure team, letting separate GPU instances share cached state directly with each other over RDMA instead of each maintaining an isolated cache. Both AMD and Nvidia backing the same KV cache start-up, alongside a cloud provider like CoreWeave, signals how central this specific bottleneck has become across competing hardware camps.
None of this requires an enterprise to rip out existing infrastructure. Tensormesh’s software sits on top of GPUs customers already own, whether AMD, Nvidia, or a mix, and the AMD integration specifically targets teams already running MI-series hardware who want more model capacity without another procurement cycle. The pitch is straightforward: The memory was already there, it just needed somewhere to go when the GPU wasn’t using it.
What do we think?
The real story isn’t the benchmark numbers, it’s that AMD, Nvidia, and CoreWeave all put money into the same small KV cache company within months of one another. That confirms memory management, not raw compute, is the current bottleneck in inference economics. For CIOs, Tensormesh defers a hardware refresh. For silicon teams, memory orchestration just became a competitive layer.
Inflection point
Three GPU-adjacent investors—AMD Ventures, NVentures, and CoreWeave—backing the same KV cache start-up marks an inflection point: Memory orchestration software now counts as strategic infrastructure, not a side optimization. When rival chipmakers fund the same company to make their hardware use less memory, the constraint has shifted from compute to memory bandwidth. Watch whether Nvidia ships an equivalent to AMD’s Live Context Virtualization; if it does, KV cache virtualization becomes a layer in every inference stack, not a Tensormesh specialty.
YOU LIKE THIS KIND OF STUFF? WE HAVE LOTS MORE. TELL YOUR FRIENDS, WE LOVE MEETING NEW PEOPLE.