Get the newsletter
Google · AI / ML · Filed May 16, 2025 · USPTO published Nov 19, 2026 · This week's release

Google Patents AI Dubbing That Keeps a Speaker's Voice and Follows the Video

Google has filed a patent application for AI dubbing that watches the video while it generates translated speech, aiming to keep the speaker's own voice and match the timing on screen.

An AI system generates foreign language speech that keeps the speaker's original voice and matches the video. Drawing from patent filing US 2026/0368452 A1.
An AI system generates foreign language speech that keeps the speaker's original voice and matches the video.All 14 drawings ↓
Publication number
US 2026/0368452 A1(US20260368452A1)
Applicant
Google LLC
Filed · Published
May 16, 2025 · Nov 19, 2026
Inventors
Michelle Dana Tadmor-Ramanovich, Alon Levkovitch, Daniel Walevski, Anja Hauth, Julian Emilio S. Salazar
Status when we published
Waiting for an examiner
Document
20 claims
US classification
704/2
Examiner
LEE, EUNICE SOMIN (Art Unit 2656)

What Google's AI dubbing system actually does

Imagine watching a cooking show filmed in Korean, but hearing it in English, in the host's own voice, with the words landing right when her mouth moves. Today that kind of dubbing usually means translators, voice actors and sound engineers re-recording every line by hand.

Google's patent application describes a computer system that does the job automatically. It looks at three things: the text of the dialogue, the video itself, and a short sample of the speaker's voice (the filing mentions clips as short as 1.5 seconds). It then builds the new audio from scratch, starting from random static and cleaning it up step by step until speech emerges.

Because the picture guides the sound, the pacing and pauses can follow what is happening on screen, and the voice should stay recognizably the same person, just in a new language.

From the filing · CLAIM 1
… processing, by a diffusion model operating on the computing system, the textual embedding, the video embedding, the one or more speaker embeddings, and the noised latent representation to produce a noise prediction …

Translation: An AI diffusion model analyzes the text, video, and speaker profile to predict and clear out unwanted audio noise.

How the model cleans noise into translated speech

Claim 1 describes a three-stage pipeline. First, the system gathers three inputs, each turned into an embedding (a compact list of numbers that summarizes something so a model can work with it): one for the text of the dialogue, one for the video frames, and one or more for the speaker's voice.

Second, a diffusion model (an AI model trained to reverse the process of adding noise to data) starts from a block of random noise in compressed form, called a latent representation. It predicts which part is noise, and that prediction is removed to leave a cleaner version. The filing says this can repeat over many steps, where more steps can mean higher quality but slower processing.

Third, a decoder turns the cleaned-up data into a spectrogram (a picture of sound, with time on one axis and pitch on the other) in the second language, and a vocoder converts that picture into audible speech.

A few details from inside the model:

  • One attention layer checks the video and another checks the text, so the output stays tied to both.
  • The speaker's voice adjusts how every layer behaves, through learned scale and shift settings.
  • The language model, video encoder, speaker encoder and diffusion model are trained together as one system.
  • The description says the approach can skip a separate step that predicts how long each sound lasts, since timing cues come from the video.
From the filing · THE ABSTRACT
… decoding, by a latent decoder operating on the computing system, the denoised latent representation to produce an output spectrogram representative of audio data in a second language …

Translation: The system translates the cleaned data into a visual map of spoken words in the target foreign language.

Why automatic dubbing matters to what you watch

If you watch foreign-language shows or online courses, you know the usual dubbing problems: a stranger's voice and speech that does not line up with the mouth. The filing aims to keep the speaker's vocal character across languages and to match speech to the picture without a separate lip-animation step or heavy manual tweaking.

For video platforms, the pitch is cost and speed. The background section calls traditional dubbing labor-intensive and costly, which limits how much content gets translated, and it names streaming services and online education as examples. The patent says fewer processing steps could let a system dub more videos in the same time. That is a claim in an application, not a tested result, and the filing names no product.

This is the 70th filing from Google in voice and speech AI we've tracked since May, a topic that already includes splitting sound into speech and noise and preserving tone in live translation.

Source: US 2026/0368452 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 14 sheets

14 drawing sheets from US 2026/0368452 A1 · click any drawing to enlarge

How the system generates translated speech matched to a speaker, drawing sheet 1 from US 2026/0368452 A1
Sheet 1 How the system generates translated speech matched to a speaker
Converting text, video, and audio into inputs for dubbing, drawing sheet 2 from US 2026/0368452 A1
Sheet 2 Converting text, video, and audio into inputs for dubbing
Inside the AI model that creates natural dubbed audio, drawing sheet 3 from US 2026/0368452 A1
Sheet 3 Inside the AI model that creates natural dubbed audio
Steps for turning text and video into synchronized speech, drawing sheet 4 from US 2026/0368452 A1
Sheet 4 Steps for turning text and video into synchronized speech
Training an AI model using feedback on its predictions, drawing sheet 5 from US 2026/0368452 A1
Sheet 5 Training an AI model using feedback on its predictions
A general AI setup that transforms raw inputs into outputs, drawing sheet 6 from US 2026/0368452 A1
Sheet 6 A general AI setup that transforms raw inputs into outputs
How the AI reads steps in order to predict results, drawing sheet 7 from US 2026/0368452 A1
Sheet 7 How the AI reads steps in order to predict results
Merging text, audio, and video for the AI, drawing sheet 8 from US 2026/0368452 A1
Sheet 8 Merging text, audio, and video for the AI
Show the other 6 sheets
Software tools used to build and optimize new AI models, drawing sheet 9 from US 2026/0368452 A1
Sheet 9 Software tools used to build and optimize new AI models
Steps for training and refining an AI model using feedback, drawing sheet 10 from US 2026/0368452 A1
Sheet 10 Steps for training and refining an AI model using feedback
Cloud servers handling user requests with an AI model, drawing sheet 11 from US 2026/0368452 A1
Sheet 11 Cloud servers handling user requests with an AI model
Devices and servers connecting over the internet, drawing sheet 12 from US 2026/0368452 A1
Sheet 12 Devices and servers connecting over the internet
Separate apps running their own AI models on one device, drawing sheet 13 from US 2026/0368452 A1
Sheet 13 Separate apps running their own AI models on one device
Apps sharing device data through a central AI layer, drawing sheet 14 from US 2026/0368452 A1
Sheet 14 Apps sharing device data through a central AI layer
Patent filing page

Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.