Get the newsletter
Microsoft · AI / ML · Filed Aug 8, 2025 · USPTO published Aug 13, 2026 · Aug 13 release

Microsoft Patents a Way to Clone Anyone's Voice Into a Foreign Language

Microsoft has patented an AI system that can take a short recording of someone's voice and use it to speak an entirely different language, keeping that person's vocal character intact, even if the system has never heard that person speak before.

Cross-lingual neural codec architecture mapping speech tokens and text prompts to generate personalized foreign speech output. Drawing from patent filing US 2026/0237379 A1.
Cross-lingual neural codec architecture mapping speech tokens and text prompts to generate personalized foreign speech output.All 10 drawings ↓
Publication number
US 2026/0237379 A1(US20260237379A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Aug 8, 2025 · Aug 13, 2026
Inventors
Zhuo CHEN, Yu WU, Long ZHOU, Shujie LIU, Yanqing LIU, Huaming WANG, Jinyu LI, Lei HE, Sheng ZHAO, Furu WEI, Chengyi WANG, Sanyuan CHEN, Ziqiang ZHANG
Status when we published
Waiting for an examiner
Document
15 claims
US classification
704/258
Parent application
National Stage Entry of PCT/CN2023/079256 (filed Mar 2, 2023)(PCTCN2023079256)

How Microsoft's voice-cloning AI crosses language barriers

Ever wished you could hear a dubbed film where the actor's actual voice, not a stranger's, reads the translated lines? That's roughly the problem Microsoft is filing a patent to solve.

The system works by giving an AI three things: a short audio clip of the person whose voice you want to copy, a sentence written in their original language (to anchor the voice), and a sentence written in a different language (the one you want the output in). The AI figures out what makes that person's voice sound like them, then speaks the foreign text in that same voice, even though the system was never trained on that speaker specifically.

The key detail is the phrase "zero-shot": you don't have to pre-register or record hours of training audio. A single short sample is enough for the system to do its job with a voice it has never encountered.

From the filing · CLAIM 1
… obtaining a speech sample comprising audio data from a target speaker, wherein the target speaker is an unseen target speaker such that no audio data from the target speaker was included in the text-to-speech training dataset …

Translation: The system can mimic a person's voice even if it has never heard them speak before.

How the model maps a voice sample to a new language

The patent describes a neural codec language model (a type of AI that treats audio the same way a text AI treats words, by breaking sound into small discrete units called "tokens") trained on bilingual pairs of speech recordings and transcripts.

At inference time, the model receives three inputs:

  • A short audio clip of the target speaker, someone the model has never seen in training.
  • A first-language text prompt, a written sentence in the speaker's native language, which gives the model a reference point for matching voice characteristics.
  • A second-language text prompt, the actual sentence the model needs to speak out loud in a different language.

The model uses the audio clip to extract a kind of voice fingerprint, then generates audio tokens for the second-language text, conditioned on that fingerprint. The result is synthesized speech in the target language that preserves the speaker's pitch, rhythm, and timbre.

The "zero-shot" label means no fine-tuning or speaker registration is needed beforehand. The model generalizes from bilingual speech transcription pairs it saw during training to handle novel speakers at runtime, which is the genuinely hard engineering problem here.

From the filing · THE ABSTRACT
… generating a personalized speech output based on the inputs and by at least converting the second text prompt in the second language using a synthesized voice of the target speaker based on the speech sample from the target speaker.

Translation: The technology creates a recording of the person speaking a new language using their own voice.

What this means for translation and synthetic voice tools

Voice dubbing today is expensive and slow: human voice actors re-record lines in each target language, and even high-end AI dubbing products usually substitute a generic synthetic voice rather than preserving the original speaker's character. A system that does this automatically from a short sample could change the economics of localization for video content, corporate communications, and accessibility tools.

The broader context is that voice cloning technology is moving fast across the industry, and Microsoft's filing focuses on the cross-lingual case, a harder and less-addressed variant of the problem. The approach leans on codec language models, an architecture Microsoft has invested in heavily through its research partnership with OpenAI and its own speech teams. Readers tracking where this area of AI is heading can find the latest Big Tech patents in voice synthesis and speech AI filed alongside this one.

Source: US 2026/0237379 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 10 sheets

10 drawing sheets from US 2026/0237379 A1 · click any drawing to enlarge

System architecture for speech synthesis, drawing sheet 1 from US 2026/0237379 A1
Sheet 1 System architecture for speech synthesis
Neural codec language modeling pipeline, drawing sheet 2 from US 2026/0237379 A1
Sheet 2 Neural codec language modeling pipeline
Audio encoder decoder quantization process, drawing sheet 3 from US 2026/0237379 A1
Sheet 3 Audio encoder decoder quantization process
Conditional codec language modeling workflow, drawing sheet 4 from US 2026/0237379 A1
Sheet 4 Conditional codec language modeling workflow
Cross-lingual personalized speech generation model, drawing sheet 5 from US 2026/0237379 A1
Sheet 5 Cross-lingual personalized speech generation model
Quantization and phonemization training structure, drawing sheet 6 from US 2026/0237379 A1
Sheet 6 Quantization and phonemization training structure
Zero-shot cross-lingual text-to-speech application, drawing sheet 7 from US 2026/0237379 A1
Sheet 7 Zero-shot cross-lingual text-to-speech application
Flowchart for multi-language voice cloning, drawing sheet 8 from US 2026/0237379 A1
Sheet 8 Flowchart for multi-language voice cloning
Show the other 2 sheets
Microsoft Patents a Way to Clone Anyone’s Voice Into a Foreign Language patent drawing, sheet 9
Sheet 9
Microsoft Patents a Way to Clone Anyone’s Voice Into a Foreign Language patent drawing, sheet 10
Sheet 10
Patent filing page