Google Patents AI Dubbing That Keeps a Speaker's Voice and Follows the Video
Google has filed a patent application for AI dubbing that watches the video while it generates translated speech, aiming to keep the speaker's own voice and match the timing on screen.

- Publication number
- US 2026/0368452 A1(US20260368452A1)
- Applicant
- Google LLC
- Filed · Published
- May 16, 2025 · Nov 19, 2026
- Inventors
- Michelle Dana Tadmor-Ramanovich, Alon Levkovitch, Daniel Walevski, Anja Hauth, Julian Emilio S. Salazar
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 704/2
- Examiner
- LEE, EUNICE SOMIN (Art Unit 2656)
What Google's AI dubbing system actually does
Imagine watching a cooking show filmed in Korean, but hearing it in English, in the host's own voice, with the words landing right when her mouth moves. Today that kind of dubbing usually means translators, voice actors and sound engineers re-recording every line by hand.
Google's patent application describes a computer system that does the job automatically. It looks at three things: the text of the dialogue, the video itself, and a short sample of the speaker's voice (the filing mentions clips as short as 1.5 seconds). It then builds the new audio from scratch, starting from random static and cleaning it up step by step until speech emerges.
Because the picture guides the sound, the pacing and pauses can follow what is happening on screen, and the voice should stay recognizably the same person, just in a new language.
… processing, by a diffusion model operating on the computing system, the textual embedding, the video embedding, the one or more speaker embeddings, and the noised latent representation to produce a noise prediction …
Translation: An AI diffusion model analyzes the text, video, and speaker profile to predict and clear out unwanted audio noise.
How the model cleans noise into translated speech
Claim 1 describes a three-stage pipeline. First, the system gathers three inputs, each turned into an embedding (a compact list of numbers that summarizes something so a model can work with it): one for the text of the dialogue, one for the video frames, and one or more for the speaker's voice.
Second, a diffusion model (an AI model trained to reverse the process of adding noise to data) starts from a block of random noise in compressed form, called a latent representation. It predicts which part is noise, and that prediction is removed to leave a cleaner version. The filing says this can repeat over many steps, where more steps can mean higher quality but slower processing.
Third, a decoder turns the cleaned-up data into a spectrogram (a picture of sound, with time on one axis and pitch on the other) in the second language, and a vocoder converts that picture into audible speech.
A few details from inside the model:
- One attention layer checks the video and another checks the text, so the output stays tied to both.
- The speaker's voice adjusts how every layer behaves, through learned scale and shift settings.
- The language model, video encoder, speaker encoder and diffusion model are trained together as one system.
- The description says the approach can skip a separate step that predicts how long each sound lasts, since timing cues come from the video.
… decoding, by a latent decoder operating on the computing system, the denoised latent representation to produce an output spectrogram representative of audio data in a second language …
Translation: The system translates the cleaned data into a visual map of spoken words in the target foreign language.
Why automatic dubbing matters to what you watch
If you watch foreign-language shows or online courses, you know the usual dubbing problems: a stranger's voice and speech that does not line up with the mouth. The filing aims to keep the speaker's vocal character across languages and to match speech to the picture without a separate lip-animation step or heavy manual tweaking.
For video platforms, the pitch is cost and speed. The background section calls traditional dubbing labor-intensive and costly, which limits how much content gets translated, and it names streaming services and online education as examples. The patent says fewer processing steps could let a system dub more videos in the same time. That is a claim in an application, not a tested result, and the filing names no product.
This is the 70th filing from Google in voice and speech AI we've tracked since May, a topic that already includes splitting sound into speech and noise and preserving tone in live translation.
Source: US 2026/0368452 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 14 sheets
14 drawing sheets from US 2026/0368452 A1 · click any drawing to enlarge
Show the other 6 sheets

No comments yet