Nvidia Patents an Adaptive Memory-Fetching System to Cut GPU Power Draw

By Patentlyze Team · Updated Jul 21, 2026

Every time a GPU's processing threads grab the same chunk of data from memory separately, that's wasted work and wasted power. Nvidia's new patent describes a system that teaches the chip to be smarter about sharing that data across threads automatically.

What Nvidia's cache-fetching patent actually does

Imagine dozens of workers in an office all walking to the filing cabinet individually to pull the same folder. Now imagine one person grabs the folder and shares it with everyone who needs it at once. That's essentially what Nvidia is trying to do inside its graphics chips.

A GPU (graphics processing unit) runs thousands of tiny calculation threads simultaneously. When many of those threads need the same data, the conventional approach has each one fetching its own copy from a fast but small memory area called a cache. Nvidia's patent describes a system that picks the best retrieval strategy on the fly, so data gets reused across threads instead of fetched repeatedly.

The result is fewer redundant memory reads, which means the chip does the same work with less energy. As GPUs grow larger and handle more AI and graphics tasks, that kind of efficiency gain adds up quickly.

How Nvidia's five retrieval models share cached data

The patent describes a method for cache-aware dataflow on a processor unit, meaning the chip actively chooses how it fetches blocks of data from its fast on-chip memory (the cache) based on how multiple compute threads plan to use that data.

Instead of a single fixed retrieval strategy, the system can dynamically apply one or more of five different models:

• Adaptive swizzling: reorders data in memory so threads can access it in a pattern that minimizes collisions and maximizes reuse
• Continuous rasterization: fetches data in a scan pattern that mirrors how GPUs traditionally process visual pixels, keeping related data together
• Alternating k-order: alternates the direction of data traversal to keep cache contents warm across back-to-back thread operations
• Periodic CTA synchronization: coordinates groups of compute threads (CTAs, or Compute Thread Arrays) at timed intervals so they share cache contents rather than each fetching independently
• Explicit tile eviction: actively removes specific data tiles from the cache once they are no longer needed, freeing space for the next required data

The method receives input parameters from the application specifying which algorithms to use, then applies them during execution. The underlying data is stored using a swizzle model (a rearrangement scheme that maps data to memory addresses in a way that avoids access bottlenecks).

What this means for GPU efficiency and AI workloads

GPUs have become the backbone of AI model training and inference, and the trend is toward larger chips with more processing cores and bigger caches. When cache management is inefficient, the chip wastes energy on redundant memory reads, which translates directly to higher electricity costs in data centers and shorter battery life in devices.

Nvidia explicitly notes in the patent that these optimizations become more valuable as chip size and core count grow. If this technique ships in a future GPU architecture, it could improve throughput and power efficiency for workloads you already care about, from AI inference to game rendering, without any changes to the software running on top.

Editorial commentary on a publicly published patent application. Not legal advice.