Google Patents a Speech Recognizer That Types Fast, Then Fixes Its Own Mistakes
Your phone's voice typing has to pick between fast and right. This Google patent tries to deliver both by letting a second pass fix the first draft.
What Google's two-pass speech recognizer does
Every time you dictate a text or ask your phone a question, it faces a tradeoff: show words instantly, or wait and get them right. Speed usually wins, which is how a question about the movie Serendipity can briefly appear on screen as "What year was serene released?"
Google's patent describes one speech recognition system that does both jobs. A first listener works while you talk and puts words on screen right away. A second listener then looks at the whole stretch of speech, including what came after each word, and corrects the draft once you stop.
A separate language model, trained only on written text, helps it catch rare names and unusual words. The filing says the system can run on your device or on a remote server.
How the second listener corrects the first
The system listens to audio in tiny slices called acoustic frames. A first encoder (the part that turns raw sound into a compact description a computer can work with) is causal, meaning it is blocked from peeking at audio that comes later. That lets it keep up with you in real time.
A second encoder takes only the first encoder's output as input, with no raw audio, and is non-causal: it gets extra right context, meaning a stretch of audio after each word. In one example it sees 5.04 seconds ahead. Both encoders feed one shared decoder, which produces a probability distribution (a ranked list of likely words or word pieces for each step).
An external language model then rescores that list. It can be trained on text alone, and the filing says that can mean more than 100 billion sentences, so it knows rare words and proper nouns. It is joined to the main model using Hybrid Autoregressive Transducer Factorization, a method that subtracts the main model's own built-in word habits so the two do not double count them.
In the examples, the encoders are a stack of 17 Conformer layers (a common speech-model building block): 15 in the first, two added on top for the second. The decoder's memory of recent words is a lookup table of about 2 million parameters, versus about 23.4 million for an LSTM-based version.
Why rare names and voicemails matter here
If you dictate texts or talk to a voice assistant, you want words to appear as you speak, and you also want Serendipity to come out as Serendipity. The filing's own example shows the draft guessing "serene released" before the corrected version lands. Voicemail transcription is the other case it describes: nothing needs to show live, so the system can wait for the full message and use more context.
Because one model handles both jobs, a company would not need to train and ship two separate speech systems. The filing says the model can be trained in a single stage. It also says the system can run on your device or on a remote server, which could cut the wait for a round trip to the cloud.
Google's 34th filing we've tracked since May in our AI models working in teams follows one on cutting audio compute costs and one routing image pieces to specialists.
Claim 1 covers a specific stack of four parts: a first listener that can only look backward, a second listener that gets extra look-ahead and sees only the first one's output, one decoder shared by both, and an outside language model that rescores the result. A system would need all four to match the claim's wording.
The claim is broad on size. It names no layer counts, no time window and no device type. Those details, such as the layer types and a set delay before the first listener starts, show up only in later claims. The claim also never says the system must run on a phone, despite the title.
The narrowing piece is the last clause. The language model has to be joined using Hybrid Autoregressive Transducer Factorization, which subtracts the main model's own built-in guess about likely words. The filing says the usual way of adding a language model tends to make the system drop words. A system that attaches one another way would not match the claim as written. That makes this a tight claim around one combination, with the extra layers of detail left to the dependent claims.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0351248 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in