Google Files Patent for Making a Small AI Think Twice Instead of Growing Bigger
Instead of making AI models bigger, Google describes sending data through the same middle section of a model several times. It says a small model can then perform like one with twice as many parameters.
What Google's looping AI model actually does
Every time you ask an AI chatbot a question, your words pass through a tall stack of processing steps, one after another. More steps tend to mean better answers, but every step has to be stored in memory. That is a big reason the largest AI models don't fit on your phone.
Google's patent application tries a different route. Instead of building more steps, it takes the middle of the stack and sends the data through those same steps again, like rereading a tricky paragraph instead of buying a longer book. The beginning and end of the stack still run just once.
Because the repeated steps are the same ones, nothing new has to be stored. The filing says a model with 1 billion adjustable settings could perform about as well as one with twice as many. Tiny add-on tweaks for each pass help every round do something slightly different.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output using a neural network with intermediate block looping.
Translation: Google patent filing details a system for running AI tasks by looping neural network blocks.
How the middle of the model gets replayed
The patent describes a neural network (software built from a stack of layers of math) split into three groups of layers, called blocks:
- First blocks: run once on the input.
- Intermediate blocks: run in a loop. Each block's output is fed back into the same block until a stopping rule is met, such as a set number of rounds or an amount of elapsed time.
- Last blocks: run once on the final looped result to produce the answer.
Every pass through a looped block reuses the same parameters (the learned numbers that make up the model), so only one copy has to be stored. This raises the network's effective depth, meaning the total number of layers the data passes through. The filing's example: a 9-layer block looped twice gives an effective depth of 18.
The filing also describes giving each loop its own small set of adaptation parameters using LoRA (low-rank adaptation, a method that stores tweaks as two small tables of numbers instead of one huge one). These tweaks are learned during fine-tuning and are far smaller than the main block.
The number of loops can be set from a target performance level, the difficulty of the task, or the memory of the device. In the filing's comparison, a Transformer-based model (the design behind most chatbots) with 1 billion parameters gained up to 13.5 percent in few-shot accuracy over a same-size model without loops. It showed no noticeable drop in perplexity (a measure of prediction quality) against a model with twice the parameters.
… repeatedly performing looping iterations until a loop termination criterion is satisfied …
Translation: The AI thinks over the problem multiple times until it decides it has a good answer.
Why this could fit bigger AI on smaller devices
The practical payoff is about memory. The filing points to running large language models on phones and lower-grade hardware, where memory is the limit. Since the loop count can be matched to a device's memory, one model design could run more rounds on a powerful server and fewer on an older phone.
There is a catch. Looping saves storage, but the filing's own comparison shows the looped model doing the same amount of computing as the larger model it imitates. Answers should not arrive faster, and battery cost matters on a phone. The filing's demonstrations are also mostly number puzzles and lookup tasks, so how well this carries over to everyday chat, photos or voice is left open.
Google's 58th filing we've tracked since May in our on-device AI privacy watch continues a pattern seen in its ad data isolation patent and location-hiding phone AI filing.
Memory is the wall that keeps big AI models off ordinary phones, and it is a hard wall: a model either fits in the memory a device has or it doesn't. Google goes at that wall directly by cutting how much has to be stored, and the target fits the problem well.
The price is time. The patent's own comparison shows the looped model doing the same amount of computing as the larger model it imitates, so answers should not arrive faster. You trade storage for effort, which suits a phone with little memory and some patience.
The evidence in the filing leans on number puzzles and lookup games, plus one headline figure of 13.5 percent. That is enough to show the idea works in a lab. Whether it holds up on the messy, open-ended questions people actually type is still open.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0320390 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in