Get the newsletter
Google · AI / ML · Filed May 15, 2026 · USPTO published Nov 19, 2026 · This week's release

Google Files Patent for Shrinking the Example Lists AI Models Reread Every Time

Every time you hand an AI chatbot a stack of examples, it has to reread the whole stack for each new question. Google's patent application describes reading the stack once, saving a compact summary, and reusing that instead.

A setup that shrinks long lists of sample answers and stores them in memory so an AI can quickly answer new questions. Drawing from patent filing US 2026/0368561 A1.
A setup that shrinks long lists of sample answers and stores them in memory so an AI can quickly answer new questions.All 14 drawings ↓
Publication number
US 2026/0368561 A1(US20260368561A1)
Applicant
Google LLC
Filed · Published
May 15, 2026 · Nov 19, 2026
Inventors
Fnu Devvrit, Inderjit Singh Dhillon, Varun Yerram, Nilesh Gupta, Cho-Jui Hsieh, Alec Michael Go, Prateek Jain, Liqian Peng, Jay Yagnik, Venkata Aditya Kusupati, Praneeth Kumar Netrapalli
Status when we published
Waiting for an examiner
Document
20 claims
US classification
706/52

What Google's example-compression idea actually does

Imagine you hire an assistant and, before every single question you ask, you make them reread a thick binder of sample answers so they know your style. It works, but it is slow, and the binder only gets thicker as you add more samples.

That is roughly how AI chatbots handle the examples you put in a prompt. Google's filing describes boiling the binder down just once into a short cheat sheet, saving it, and handing the assistant only that sheet whenever a question comes in.

One AI model reads the examples, and another squeezes what it learned into a small package that stays the same size. Later, when you ask something new, the AI checks the package instead of the whole pile. That means less memory used and, in theory, quicker answers.

From the filing · CLAIM 1
… processing, by the computing system, the encoded representation and the set of memory tokens with a second machine-learned model to generate a compressed representation of the plurality of exemplars; wherein the compressed representation has a reduced datasize compared to the set of prefix tokens; …

Translation: An AI shrinks down a list of examples so it takes up less space in memory.

How three AI models shrink a pile of examples

The patent covers a way to turn a long list of examples (the filing calls them exemplars) into a much smaller stored summary. Giving a model worked examples inside the prompt is called in-context learning: the model picks up the pattern from the examples without being retrained.

Here is the pipeline described in the claims:

  • The examples are broken into prefix tokens (small chunks of text, images or audio that a model reads).
  • A first model reads those tokens and produces an encoded representation, a numeric digest of what the examples say.
  • A second model combines that digest with a set of memory tokens (a small, fixed-size set of slots the summary gets squeezed into) using cross-attention, a step where one set of data picks out the most relevant parts of another. The output is the compressed representation.
  • That result is stored, for example in a cache, and a third model uses it as a conditioning signal (background guidance) when it processes a new question.

Because the memory tokens stay a fixed size, the summary stays the same size no matter how many examples go in. The filing also describes using text, images or audio, using the same model or two different models for the first two steps, and tuning the pieces over repeated rounds of training.

From the filing · THE ABSTRACT
… wherein the compressed representation has been generated by performing cross-attention in a first machine-learned model on a set of prefix tokens or an encoded representation thereof relative to a set of memory tokens or an intermediate-processing representation thereof; …

Translation: The system squeezes the data by comparing the examples against a special set of memory tokens.

Why shorter prompts could mean cheaper, faster AI

If you have ever pasted a long set of samples into a chatbot, that cost shows up as slower replies and, for companies, bigger computing bills. The filing says rereading many examples on every request eats processor time and memory, especially on devices with limited hardware. A compact summary prepared ahead of time could cut that cost and make it practical to give a model many more examples.

The idea is described for text, images and audio alike. For you, the payoff would be faster answers and AI features that could run on smaller devices. For Google, it means serving more requests with the same computers. The filing reports no test numbers, so how much is saved is still unproven.

This is the 97th Google filing we've tracked in our AI assistant and agent coverage since May, building on ideas like loading only needed data and specialist helpers behind one chat box.

Source: US 2026/0368561 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 14 sheets

14 drawing sheets from US 2026/0368561 A1 · click any drawing to enlarge

Storing compressed tutorial examples for faster AI answers, drawing sheet 1 from US 2026/0368561 A1
Sheet 1 Storing compressed tutorial examples for faster AI answers
Three connected AI models shrink examples and answer user requests, drawing sheet 2 from US 2026/0368561 A1
Sheet 2 Three connected AI models shrink examples and answer user requests
Passing compressed examples across matching stages in three AI models, drawing sheet 3 from US 2026/0368561 A1
Sheet 3 Passing compressed examples across matching stages in three AI models
Internal parts that condense reference examples into compact memory, drawing sheet 4 from US 2026/0368561 A1
Sheet 4 Internal parts that condense reference examples into compact memory
Teaching an AI model to improve using performance feedback, drawing sheet 5 from US 2026/0368561 A1
Sheet 5 Teaching an AI model to improve using performance feedback
Converting incoming user requests into finished AI results, drawing sheet 6 from US 2026/0368561 A1
Sheet 6 Converting incoming user requests into finished AI results
Breaking an incoming question into pieces to generate an answer, drawing sheet 7 from US 2026/0368561 A1
Sheet 7 Breaking an incoming question into pieces to generate an answer
Combining text, images, and other media into one input stream, drawing sheet 8 from US 2026/0368561 A1
Sheet 8 Combining text, images, and other media into one input stream
Show the other 6 sheets
Software tools engineers use to build and shrink AI models, drawing sheet 9 from US 2026/0368561 A1
Sheet 9 Software tools engineers use to build and shrink AI models
Steps for training and streamlining an AI with user feedback, drawing sheet 10 from US 2026/0368561 A1
Sheet 10 Steps for training and streamlining an AI with user feedback
How a cloud server runs AI to answer requests, drawing sheet 11 from US 2026/0368561 A1
Sheet 11 How a cloud server runs AI to answer requests
Connected computers and servers sharing data to train AI systems, drawing sheet 12 from US 2026/0368561 A1
Sheet 12 Connected computers and servers sharing data to train AI systems
A device running separate AI models inside each individual app, drawing sheet 13 from US 2026/0368561 A1
Sheet 13 A device running separate AI models inside each individual app
Multiple apps sharing a single central AI brain, drawing sheet 14 from US 2026/0368561 A1
Sheet 14 Multiple apps sharing a single central AI brain
Patent filing page

More in AI assistant & agent

All 436 AI assistant & agent patents →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.