Google Patents a Chip Layout That Splits AI Work Across Shared Memory Tiles
Google has filed a patent for a custom chip architecture that lets multiple processing units share the same memory pool while running AI calculations in parallel. It's a structural bet on how to make AI hardware faster without wasting memory.
What Google's shared-memory AI chip design actually does
Imagine a busy restaurant kitchen where every cook has their own fridge. If one cook needs an ingredient the other has, they have to wait or duplicate supplies. Now imagine all the cooks share one central fridge with a smart dispatcher handing out ingredients as needed. Google's patent describes something like that second kitchen, but for AI chips.
When a chip runs an AI model, it needs to feed huge amounts of data (called inputs and weights) to the parts of the chip doing the actual math. Normally, each processing unit manages its own memory, which wastes space and time. Google's design uses a shared memory block that multiple processing units all pull from at once, with dedicated logic deciding who gets what and when.
The result is that the chip can run many calculations at the same time without each unit hoarding its own copy of the data. For you, that could eventually mean faster AI responses in Google products, or more powerful AI features running directly on a device without needing the cloud.
How the unified memory and arbitration logic divide AI work
The patent describes an integrated circuit (a custom chip) built around units called super tiles. Each super tile contains:
- A unified memory: a single shared storage block holding the inputs and weights (the learned parameters) for a given layer of a neural network.
- Multiple compute tiles: smaller processing units, each running its own compute thread (a parallel stream of calculations).
- Arbitration logic: dedicated circuitry that acts as a traffic controller, deciding which compute tile gets which slice of data from the unified memory at any given moment.
The key technical idea is that two or more compute threads can process different sub-partitions of an input tensor (think: different chunks of the same dataset) while drawing from the same memory, rather than each keeping a private copy. The arbitration logic passes inputs and weights out to the compute tiles and then collects the results back into unified memory once the layer's calculations finish.
This approach targets a classic bottleneck in AI chip design: memory bandwidth. Moving data between separate memory banks eats time and power. By centralizing storage at the super tile level and managing access with dedicated hardware logic, Google aims to cut that overhead significantly.
What this means for Google's in-house AI chip ambitions
Google designs its own AI chips, called TPUs (Tensor Processing Units), specifically to run the machine-learning workloads powering Search, Translate, Gemini, and other services. This patent sits squarely in that lineage. A more efficient memory-sharing architecture could let future TPUs handle larger AI models at lower cost per calculation, which matters enormously when you're running billions of queries a day.
For users, the downstream effect is faster and cheaper AI. For the industry, it signals that Google is still investing heavily in custom silicon rather than relying on off-the-shelf chips from Nvidia or others. The architectural choices described here, shared tile memory plus hardware arbitration, are the kind of low-level decisions that compound into large performance advantages over many generations of chip design.
This is genuinely interesting chip architecture work, not a flashy consumer feature but the kind of foundational engineering that determines whether Google's AI infrastructure gets cheaper to run over time. The shared-memory-with-arbitration approach addresses a real bottleneck. Whether this specific design makes it into a shipping TPU is unknowable from the filing alone, but the problem it solves is real and consequential.
The drawings
5 drawing sheets from US 2026/0220449 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.