Google
Google · AI / ML · Filed Jul 7, 2026 · USPTO published Nov 12, 2026 · This week's release
Google Patents a Cheaper Way for AI to Weigh Every Word of a Long Document
The AI behind chatbots gets very expensive as the text it reads gets longer, because it compares every word with every other word. Google's patent describes a way to keep that full view while doing far less of the work.
- Publication number
- US 2026/0359561 A1(US20260359561A1)
- Applicant
- Google LLC
- Filed · Published
- Jul 7, 2026 · Nov 12, 2026
- Inventors
- Hanjun Dai, Bo Dai, Hongyu Ren, Dale Eric Schuurmans, Zihang Dai, Mengjiao Yang
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 706/15
- Parent application
- Continuation of 17/860,691 (filed Jul 8, 2022)(17860691)
What Google's Combiner attention trick does
Every time you hand a chatbot a long report and ask for a summary, the AI behind it compares each piece of the text with every other piece. That is thorough, but the work balloons. Make the text ten times longer and the effort grows about a hundredfold, which is why huge inputs like high-resolution images or raw audio get costly fast.
Google's patent describes a shortcut. Instead of checking every word one by one, the model looks closely at a chosen few and takes a quick look at the rest as bundles. It is like skimming chapter summaries before deciding which pages to reread. Once it knows a bundle matters, it works out how much each word inside that bundle counts.
The payoff, according to the filing, is that every word can still influence every other word, while the total work grows much more slowly. Google's example version of the idea is called Combiner.
… apply a structured attention pattern to the sequence of input data elements to generate a sequence of output data elements; wherein, for each input data element in the sequence of input data elements, the structured attention pattern specifies one or more locations of direct expectation and one or more groups of locations of local expectation; …
Translation: The system organizes how the AI weighs words by combining direct connections with grouped local contexts.
How each word checks groups instead of all words
Modern AI language and image models rely on attention, the step where the model decides how much each piece of the input (a word, a patch of an image) should care about every other piece. Standard attention compares all pairs, so the work grows with the square of the input length. This filing describes a replacement the document calls Combiner.
For each position, everything else is split into two kinds. A few locations get direct expectation: the model scores them one by one, as usual. The rest are sorted into groups with local expectation. For each group, the model computes a single group probability (how much this position cares about the group as a whole) and an individual score for each member (how much weight it gets inside the group).
The within-group scores do not depend on which position is asking, so they can be computed once and reused for many positions. That reuse is where the savings come from. The filing says well-chosen groupings bring the cost to roughly L log L, or L times the square root of L, where L is the input length, instead of L squared.
The filing also describes:
- Groups nested in a tree with several levels
- Fixed, log-spaced, row-and-column, or learned grouping plans
- Blending two grouping plans to improve quality
It is pitched as a drop-in replacement for existing attention layers.
Each location can attend to all other locations, either via direct attention, or through indirect attention to group representations, which are again conditional expectations of embeddings from corresponding local regions.
Translation: Every part of the text can look at every other part, either directly or by summarizing local regions to save computing power.
Why Google's patent matters for long-input AI
If you use AI that reads long documents, long conversations or large images, the cost of that reading is a big part of what limits it. Cutting the work without having the model skip parts of the input could mean cheaper runs, lower memory needs, and room for longer inputs. The filing lists text, images, audio, protein data and computer code as targets.
The patent says its example version also worked on ordinary hardware without special chips, and that tests showed strong results on image and text tasks while running faster. It is a building-block idea that would sit deep inside other products. Nothing here names a specific product, so treat it as a sign of where the efficiency work on large AI models is aimed.
Google's 93rd filing in the AI assistant and agent work we've tracked since May adds to a run that includes a phone AI that cools itself and photo-reading ad chatbot.
Source: US 2026/0359561 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
Add Patentlyze on Google
One click.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 5 sheets
5 drawing sheets from US 2026/0359561 A1 · click any drawing to enlarge
More from Google
All 1,079 Google patents →
More in AI assistant & agent
All 413 AI assistant & agent patents →
OpenAI
OpenAI Patents a Rulebook Its AI Reads Only When a Touchy Topic Comes Up
Nvidia
Nvidia Patents a Memory Shortcut That Lets Small AI Models Draw Sharper Images
Microsoft
Microsoft Patents a Shortcut for the Math That Slows AI Chatbots
Microsoft
No comments yet