A New Google Patent Lets Phone AI Compare Several Answers Before Replying
Getting an AI to try several answers and pick the best one usually takes server-sized memory. Google's patent application describes a way to do it on a phone by letting every draft share one workspace.
What Google's shared-workspace AI trick does
Imagine asking your phone's assistant to rewrite a tricky email. AI tends to do better when it drafts several versions and picks the winner, but that eats a lot of memory and processing power, which is why it mostly happens on big servers instead of in your pocket.
Google's patent application describes a shortcut. Instead of making a full separate copy of the work for every draft, the AI keeps one shared workspace plus a running index that tracks which pieces belong to which draft. The drafts branch off from the same starting point, and a masking trick keeps them from mixing together.
When the drafts are done, the AI scores them and picks the best one to show you. The goal is the quality boost of multiple tries without the memory bill that normally comes with it.
… generating, based on the dependency map, a plurality of virtual batches, each of the plurality of virtual batches comprising a discrete inference; and selecting, by the LLM, one or more of the plurality of virtual batches as a final inference.
Translation: The AI creates multiple potential answers and then picks the best one to show you.
How the index keeps each draft answer separate
The filing starts with how an AI language model (LLM) reads your request. It breaks the request into tokens, small chunks of words or, for images and audio, numeric stand-ins called soft tokens. The opening chunk is the prefix sequence. It is read all at once, with every token looking at every other token (the patent calls this non-causal attention).
Next, the model builds a dependency map: a simple list pairing each token's position with the earlier token it follows. A value of -1 means the token starts a fresh chain. Each chain through the map is a virtual batch, meaning one candidate answer that exists as a path through shared memory instead of as its own full copy.
To keep candidates apart, the filing uses attention masks (grids that say which tokens may look at which):
- A square mask over the prefix, so it is read in both directions.
- A triangular mask over later tokens, so each one sees only itself and what came before.
- Blank, masked spots that stop one candidate from peeking at another.
The model can then compare candidates using confidence scores and pick the highest, or show nothing if no candidate clears a threshold. The filing also describes splitting a long prefix into chunks, tracking the relationships through a feed-forward layer (a processing stage inside the model), and feeding in images, audio, video or robot motion.
Generating the plurality of virtual batches may include applying an attention mask having a non-causal mask portion corresponding to the prefix sequence.
Translation: It uses a special filtering technique to help the AI consider different paths of thought at the same time.
Why this could matter for AI on your phone
If you use AI on your phone, the practical stake is better answers without leaning on a server. The filing says multiple-answer approaches can cost too much for phones, watches and headsets, which pushes them onto big servers. That can mean slow replies, or no access at all when your device isn't connected. Sharing one memory space is meant to lower that cost.
The patent application lists smartphones, smartwatches, earbuds, AR glasses and cars as possible homes for the technique. It also covers image, audio and video input, so the savings could apply well beyond typed text. Examination is currently on hold at the patent office, and the application says nothing about when, or whether, any Google product would use it.
Google's 59th filing we've tracked in our on-device AI privacy work since May builds on ideas like keeping a small model efficient and ad AI that guards user data.
The filing describes software running on ordinary phone processors, so no new chip has to exist first. What does have to exist is a model built to read its opening input all at once, which the filing calls a prefix language model, plus an inference engine that can read and maintain the index.
That makes the shortest route to a product an engineering project inside an on-device AI runtime. Nothing here requires new hardware or a new kind of model training.
You would never see this feature directly. You would just notice that a phone assistant gives better answers, or gives them faster, than it otherwise could. That is a modest payoff, but a believable one, and it is a plumbing improvement that sits close to shippable.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0320364 A1 · click any drawing to enlarge
Show the other 4 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in