Google's New Patent Covers a Shortcut That Helps AI Text Tools Run Faster
Google has filed a patent application for giving AI text tools a cheat sheet of common word pairs. Its own filing says some language tasks could run about twice as fast as a result.
What Google's N-grammer layer actually does
Every time you ask an AI tool to summarize an article or translate a sentence, it has to work out what each word means from scratch, even for phrases it has handled millions of times. That takes computing power, and computing power means waiting.
Google's patent application adds a quick step before the main AI does its heavy work. The step sorts each word into one of a few thousand buckets of similar meaning, then pairs it with the word just before it. Common pairs get a ready-made entry in a lookup table, so the AI gets a head start on context instead of recalculating it every time.
The filing says this can make some language tasks about twice as fast, while keeping accuracy at or above the best existing systems.
… generate a discrete latent representation of the sequence of input embeddings that comprises, at each of the plurality of input embedding positions, a respective discreate index for the respective input embedding at the input embedding position …
Translation: It converts text data into a series of shorthand numerical codes to process words more efficiently.
How the layer turns word pairs into lookups
The patent covers a neural network (software loosely modeled on how brains learn patterns) built from N-grammer layers and attention layers. Attention layers are the part of modern AI text systems that decides which words in a passage matter to which others. The N-grammer layer is the new piece, and it prepares the input for them.
Text arrives as a sequence of embeddings (lists of numbers standing in for each word). For each one, the layer does the following:
- Finds the closest entry in a codebook, a learned list of code words (the description gives 4,096 as an example size), and records that entry's ID.
- Builds an n-gram index by combining that ID with the IDs at the preceding positions. An n-gram is just a run of consecutive items; a two-word run is a bi-gram.
- Uses the index to fetch a stored n-gram embedding from a hash table (a quick-reference list) whose values are learned during training.
- Joins the fetched embedding with the original one to form the output.
The layer can run several heads (parallel copies) that use their own codebooks or share one. The claims also describe layer normalization (keeping the numbers on a similar scale) before the two are stuck together end to end. Because these lookups are sparse, the description says cost does not grow in step with the number of stored parameters.
… generate a discrete latent representation of the sequence of input embeddings by using a learned product quantization codebook …
Translation: The system uses a special codebook to translate text into compressed numerical tokens.
Why faster, cheaper AI answers matter to you
If the approach works as described, the payoff lands on your side of the screen as shorter waits. Summaries, translations and typed answers could come back faster, and training these systems could take less computing time and power, which matters because every AI service pays that bill. The filing also lists uses well beyond chat, including speech recognition, text to speech, health record predictions and image generation.
The application is a continuation of one Google filed in 2022, and it names Rohan Anil and Aurko Roy as inventors. It does not tie the idea to any named product. The speed figures (about twice as fast on some language tasks, with accuracy that matches or beats the best existing systems) come from the filing's own description, and no outside test is cited.
Google's 19th filing we've tracked in the AI chip wars since July follows its work on memory that stops growing and predictive overheating chips.
If this ever reaches a product you use, you would notice it as a shorter wait. A summary, a translation or a typed answer would show up sooner, and the service behind it would cost less to run. That is the whole pitch, and it is a decent one.
You will not see a new button or a new feature. The twice-as-fast figure comes from Google's own description and applies to some examples, so read it as a best case.
The filing is a continuation of a 2022 application, which makes it a refinement of an older idea. Whether any of it runs inside a Google product today, the patent does not say.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
3 drawing sheets from US 2026/0320291 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in