Google Patents a Way to Cut Wasted Memory Moves Inside AI Chips

By Patentlyze Team · Updated Jul 10, 2026

Every time an AI model reorganizes its data mid-computation, the chip has to shuffle numbers around in memory. Google has filed a patent for a smarter schedule that cuts out the pointless shuffles before they even start.

What Google's tensor reshape fix actually does

Imagine a warehouse where workers are moving boxes from shelf to shelf, but some of them are caught in a loop: Box A goes to Bay 2, which sends it to Bay 7, which sends it right back to Bay 2. Nothing gets done, space gets wasted, and everyone is stuck waiting. That's roughly what can happen inside an AI chip when a neural network needs to "reshape" its data.

In AI, a reshape operation is like reorganizing a spreadsheet from 10 rows by 10 columns into 100 rows by 1 column. The numbers stay the same; only the arrangement changes. But on a chip with many small memory banks, figuring out which bank hands data to which other bank can create circular routes where data loops around instead of arriving at its destination.

Google's patent describes a system that spots those circular routes before the chip starts working, removes them from the plan, and sends a cleaner set of instructions to the hardware. The result is less wasted memory space and faster execution.

How the compiler spots and removes circular data transfers

The patent covers the compilation step that happens before a neural network layer runs on a hardware accelerator (a specialized chip, like a TPU, designed specifically for AI math).

Neural networks are built from layers, and one type of layer is a reshape layer, which changes the shape of a tensor (a tensor is just a multi-dimensional array of numbers, like a 3D grid of values) without changing the values themselves. On a chip that splits its memory into many small tile memories, moving data from one shape to another requires routing chunks of that data between tiles.

Here's the core problem the patent solves: when you map out all those tile-to-tile transfers, you can end up with cyclical routes, meaning Tile A needs to send to Tile B, which needs to send to Tile C, which needs to send back to Tile A. Nothing can move until everything moves.

The patented method works in steps:

• Receive a schedule of all planned data transfers between tile memories.
• Identify any cyclical (circular) routes in that schedule.
• Remove those cycles to produce an updated, deadlock-free schedule.
• Compile that updated schedule into instructions the hardware can actually execute.

Breaking cycles before compilation means the chip never has to stall or use extra scratch memory to resolve a deadlock at runtime.

What this means for AI chip efficiency at Google's scale

At Google's scale, neural networks run on custom TPU chips across massive data centers. Even a small inefficiency in how data moves inside a chip, multiplied across billions of operations and thousands of chips, adds up to real cost in time and power. A reshape layer might seem like a minor housekeeping step, but it appears in many modern model architectures, so making it reliably efficient has a broad impact.

For you as a user, this kind of work is invisible but consequential. Faster, more memory-efficient AI inference is part of what makes Google's cloud AI services cheaper to run, which eventually flows into response times and pricing for products built on top of those services.

Editorial commentary on a publicly published patent application. Not legal advice.