Google Files Patent to Let AI Models Reuse Work They Already Did
AI models spend a lot of effort re-deriving answers they already worked out one step earlier. Google's patent application lets them copy the old answer instead.
What Google's reuse trick means for AI chatbots
You're pasting a long report into a chatbot, and the answer crawls in. Behind the scenes, the AI is comparing every word in your report with every other word, layer after layer, to work out what relates to what.
Google's patent application starts from a simple observation: much of that comparing is repeated. Its researchers found that neighboring layers of these models often reach very similar conclusions about which words matter to each other. So instead of recalculating, a later layer can borrow the earlier layer's results. Some parts of a layer reuse, and others still compute fresh.
In the tests described in the filing, models built this way matched or beat standard ones while using less computing power and memory. For you, the payoff could be the same kind of AI costing less to run.
… transforming, reusing the first computations of the first attention layer, an input to the second attention layer; generating a model output based on the second layer output.
Translation: The second layer reuses math from the first layer to save time.
How later layers borrow earlier layers' scores
The patent covers a way to build Transformers (the type of AI model behind most modern chatbots) so that one layer can reuse work another layer already did. In these models, a step called attention scores how strongly every word in the input relates to every other word. Each layer runs several attention heads, which are parallel readers that each form their own set of scores.
The claims say a second attention layer transforms its input using the first layer's computations, which can be the attention scores themselves. Those results sit in a carry-forward tensor (a stored block of numbers handed from layer to layer), and the later layer combines them with its own data instead of recalculating.
The description lays out a few ways to set it up:
- Some heads in a layer reuse old scores while the rest compute fresh ones. One example has six heads, four of them reusing.
- Every head in a layer reuses, with the first layer always computed in full.
- A reusing head blends scores from two or more earlier layers, using weights the model learns.
- Reuse is built in during training, or added later to a trained model after a test finds which layers repeat each other.
Heads that reuse skip the costliest step, so the layer needs less computing and fewer parameters (the stored numbers that make up the model).
To reduce the computational cost of self-attention-based models while achieving comparable or even superior results, example aspects of the present disclosure propose a novel architecture that reuses attention scores computed in one layer in one or multiple subsequent layers.
Translation: Recycling previous calculations makes AI models faster without losing accuracy.
Why cheaper attention matters for long documents
If you use AI on long documents, speed and price come down to this exact step: attention work grows quickly as text gets longer. Cutting some of it could let the same hardware serve more people, or handle bigger jobs with less memory. The filing says its tests matched or beat standard models while using less computing power and memory.
The tests named in the filing cover older, widely used models (BERT, T5 and a vision model called ViT), and the filing names no product. The idea also applies broadly, since it works on language, images and other inputs that pass through these layers. Companies that run these models at scale have the most to gain.
Google's 54th filing in our AI training and infrastructure coverage we've tracked since May adds to a run that includes one on image-word matching and one on sizing AI models.
Running AI text models is expensive in a specific way. Each time a model reads text, it compares every word with every other word, so doubling the length of a document roughly quadruples that work. As people paste in longer files and ask for longer answers, that bill lands on whoever runs the servers.
Google's fix is modest in size. It skips some of the repeat calculations when neighboring layers would reach nearly the same answer. That trims the bill by a fraction and leaves the pile-up with longer text in place. Still, a trim that costs little in quality, and can be added to an already-trained model, fits a problem where small savings multiply across enormous usage.
The proof so far comes from the filing's own tests on older, well-known models. The open question is how well the idea holds up in modern giant ones.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0330180 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in