Google Patents a Way to Show AI Answers Before They're Fully Written
Google has filed a patent for a system that splits an AI model's response into pieces and starts displaying them on your screen before the model has even finished writing the rest, shaving precious fractions of a second off the wait that makes chatbots feel sluggish.
How Google's chunk-by-chunk AI output actually works
You're typing a question into an AI assistant and hitting send. Instead of staring at a spinning cursor while the model generates every single word before showing you anything, the response starts appearing almost immediately, word by word or sentence by sentence.
That's the core idea in this Google patent. Their system has the AI generate a first chunk of its answer, evaluate it, and start showing it to you while it's still writing the rest. Each chunk is picked from several candidates, so the model is still trying to give you its best response rather than just rushing out the first thing it generates.
The practical upshot: conversations with AI feel faster and more natural, the way texting feels faster than waiting for someone to finish composing a full email before hitting send.
… determining, based on the LLM output, a plurality of first candidate segments for a first segment of the stream of NL based output; selecting a given first candidate segment, from among the plurality of first candidate segments for the stream of NL based output, as the first segment of the stream of NL based output …
Translation: The system generates several possible options for the next part of the answer and picks the best one to show the user.
How the LLM picks and chains each response segment
The patent describes a server-side pipeline that works in stages. When you send a message, the large language model doesn't just produce one long answer and ship it over. Instead, it generates several candidate options for the first chunk of the response, picks the best one, and sends that chunk to your device right away.
While your screen is already rendering that first piece, the model loops back and uses it as a starting point to generate candidates for the second chunk, picks the best, and sends that. This continues segment by segment until the full response is assembled.
The key technical detail is in how the selection works. The claim describes:
- The LLM processes the original input to produce a set of first candidate segments
- One candidate is chosen as the confirmed first segment
- The LLM then processes the original input plus that first segment to generate candidates for the second segment
- That chosen second segment feeds the next round, and so on
This is sometimes called speculative decoding or streaming in LLM engineering, a technique where output is pipelined rather than batched. The patent's specific framing is around selecting from multiple candidates at each stage, which adds a quality-control layer on top of simple token streaming.
In some implementations, a first segment of the stream of NL based output is selected for inclusion in the stream of NL based output as a second segment (and any subsequent segment) is being generated to reduce latency in evaluating the NL based output as a whole prior to rendering thereof.
Translation: The AI starts showing you the beginning of an answer while it is still busy calculating the rest of the text.
What this means for AI assistant response speed
For anyone using AI assistants on mobile or in latency-sensitive apps, the difference between a response that appears word-by-word versus one that arrives all at once after a two-second pause is significant. This patent targets exactly that gap, and it applies anywhere a language model's output is rendered to a user in real time, whether that's Google's Gemini products, Search's AI Overviews, or developer-facing APIs.
Claim 1 is written broadly. It covers any method of generating a stream of NL output where segments are selected from candidates and fed forward, with no restriction to a specific model architecture or product surface. That breadth means the claim, if granted, would potentially reach a wide range of streaming AI response systems that follow this select-then-stream pattern. Patentlyze publishes plain-English patent summaries across AI infrastructure filings like this one, where the real competition is often in the latency plumbing rather than the model itself.
Claim 1 is written at a level of abstraction that covers a large portion of how modern LLM streaming already works in practice: generate candidates for a chunk, pick one, use it to generate the next. If the USPTO grants this without significant narrowing, it could give Google a broad stake in a technique that other AI providers already deploy in some form. The patent does not describe a novel algorithmic breakthrough so much as a systematic, claimable framing of an engineering pattern that is increasingly standard.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0236495 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →