Google Files Patent for Reusing a Chatbot's Past Work on Similar Questions
Google's patent filing describes a chatbot that remembers its earlier thinking. When your question is close to one it has seen before, it only has to finish the last step or two.
What Google's reuse-old-work chatbot patent does
Imagine you ask a chatbot, "where is the highest mountain around the world?" A few moments later, a stranger asks nearly the same thing with a couple of words changed. Today, the AI does all of its heavy thinking again from scratch for both of you.
Google's patent describes a way to save that work. The system keeps a library of past questions, along with the AI's half-finished thinking from the steps just before it wrote each answer. When your question arrives, it checks the library for a close match.
If it finds one, it picks up the old thinking near the finish line and only runs the last step or two. The closer the match, the more it skips. The goal is a faster reply that uses less computing power and less back-and-forth over the internet.
… retrieving a precomputed intermediate layer output associated with the previously received query in the database, wherein the intermediate layer output relates to a processing of the previously received query using an intermediate layer of the generative model; …
Translation: The system grabs saved mental steps from an older question instead of starting from scratch.
How Google picks the point to resume from
A large language model (LLM, the type of AI behind chatbots) processes your question through a long stack of layers, each one refining the work of the one before. Google's filing saves the output of those layers for past questions in a query database, so that work can be reused later.
When a new question arrives, the system turns it into an embedding (a long list of numbers that captures its meaning) and measures how close it sits to each stored question. Then it picks a saved layer output based on how close the match is:
- A very close match (the filing's example is 0.95 or higher) uses the output from the second-to-last layer, so only the final layer still has to run.
- A looser match (the example is 0.8 or higher) uses the third-to-last layer, so two layers have to run.
- No match means the AI works through the question the normal way.
The saved output is combined with the new question's own representation and fed into the remaining layers to produce the answer. For a near-exact match, the filing also describes pausing the AI's ongoing work on the new question and jumping straight to the saved shortcut. The claims also cover image-based questions, and a setup where the last layers run on your device while the database sits on a remote server.
Particular intermediate model output associated with the particular historical query can be selected from all intermediate model outputs stored in association with the particular historical query, and a response to the user query can be generated based at least on the selected particular intermediate model output associated with the particular historical query.
Translation: It picks the exact past calculation it needs to build a fast answer for you.
What this could mean for chatbot speed and cost
If it works as described, popular questions could get answers sooner, and the servers behind them would do less work for each one. The filing says the savings include less computing power and less data sent back and forth between your device and a server, especially when part of the model runs on your phone or laptop.
The idea has trade-offs. The database collects questions from many users and devices, and it keeps saved work for every layer of the model. The filing says personal information can be removed before anything is stored. It also gives no measured speed or accuracy results, and it is a patent application, not a product.
Google's 43rd filing we've tracked since May sits alongside one on delaying updates mid-edit and one on radar guest detection in our growing AI assistants that remember you group.
Waiting several seconds for a chatbot reply costs you a few moments and costs the company real money. Every answer runs through a model with billions of adjustable numbers, and the filing points out that this happens hundreds or thousands of times a minute. Trimming even a slice of that work adds up fast.
The idea fits the size of that problem well. People ask the same things in slightly different words all day, and the filing's own example, two phrasings of the highest-mountain question, shows how much overlap there is. Reusing nearly finished work for a near-duplicate question skips most of the effort.
The weak spot is trust. Reusing old work means betting that two questions mean the same thing, and one changed word can flip an answer. The patent handles that with similarity cutoffs and a fallback to full processing, but it offers no measured results, so the payoff stays unproven on paper.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0350021 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in