Get the newsletter
Microsoft · AI / ML · Filed Apr 30, 2026 · USPTO published Nov 19, 2026 · This week's release

Microsoft Files Patent for a Checkup Tool That Finds Why Voice Software Mishears Words

A Microsoft patent application has a clever way to find out why voice software mishears people: have a computer read the same transcript aloud, then compare the mistakes. The gap between the real and synthetic versions points to the culprit.

A diagnostic program plays recorded and synthetic speech into a virtual assistant to uncover why it misunderstands spoken words. Drawing from patent filing US 2026/0369265 A1.
A diagnostic program plays recorded and synthetic speech into a virtual assistant to uncover why it misunderstands spoken words.All 12 drawings ↓
Publication number
US 2026/0369265 A1(US20260369265A1)
Applicant
MICROSOFT TECHNOLOGY LICENSING, LLC
Filed · Published
Apr 30, 2026 · Nov 19, 2026
Inventors
Haoxuan LI, Rui JIANG, Yang LIU, Edward C LIN, Lei SUN, Che ZHAO
Status when we published
Waiting for an examiner
Document
20 claims
US classification
704/243
Parent application
Continuation of 18/707,542 (filed May 3, 2024)(18707542)

What Microsoft's speech-recognition checkup does

You're dictating notes into a work app, and it keeps mangling the same words. Is it your accent, the noisy office, or a word the software never learned? Working that out is surprisingly hard, even for the people who set the software up.

Microsoft's patent application describes a checkup tool for that mystery. It takes a batch of real recordings with correct written transcripts, then has a computer read those same transcripts aloud in a clean synthetic voice. Both versions go through the speech recognizer, and the tool compares the mistakes. If the real recordings go wrong far more often than the clean copies, the trouble is probably the sound itself, like background noise or accents. If both go wrong on the same words, the problem is likely vocabulary or grammar.

The tool then tells the person in charge what to collect next, such as more recordings from similar settings or sample sentences for the trouble words.

From the filing · CLAIM 1
obtaining a test dataset comprising (i) audio data having a plurality of natural speech utterances and (ii) a transcription of the plurality of natural speech utterances; generating a text-to-speech dataset comprising audio data having a plurality of synthesized speech utterances based on the transcription of the plurality of natural speech utterances; …

Translation: The tool compares real human speech recordings with computer generated voice clips of the exact same words.

How real and computer-voiced audio get compared

Modern speech recognition has three parts. The acoustic model matches sounds to speech units, the language model guesses which words are likely to follow each other, and the lexicon is a dictionary of words and their pronunciations. The filing's diagnostic tool tries to work out which part is to blame for each mistake.

For the acoustic model, it runs real recordings and computer-spoken copies of the same transcripts through the system, then compares how many edits each output needs. In the filing's example, the real set needed 6,549 edits (a 12.9% word error rate) and the synthetic set needed 3,342 (6.6%). Subtract, divide by 6,549, and you get an acoustic model error ratio of 49%.

The clean synthetic audio then serves as a reference for the other two parts. Words that still fail get sorted into types:

  • Language model: words it has never seen, words with too little context, or words wrongly glued together or split apart.
  • Lexicon: words missing from the dictionary, or pronunciations that don't match.

Each part gets its own score, and the tool suggests targeted data to collect: audio from a similar scenario, sample sentences for each trouble word, or a listen-and-check of pronunciations. In one test, adding text pulled from web search results for the trouble words lowered the word error rate from 12.82% to 12.30%.

From the filing · THE ABSTRACT
Systems identify a first set of errors in the first acoustic model output and a second set of errors in the second acoustic model output. Finally, based on comparing the first set of errors and the second set of errors, an acoustic model error ratio is generated.

Translation: By comparing where the software messed up on real versus synthetic voices, the system measures its overall accuracy.

Why custom voice software may get easier to fix

The practical win is for the people who set up speech software for a specific job: a company with its own jargon, noisy work settings, or customers with strong accents. According to the filing, it can be hard to tell why a customized system underperforms, and harder still for someone without AI training to fix it. This tool turns a single accuracy number into a short to-do list: collect more audio like this, add sentences for these words, check these pronunciations.

For you, the payoff would show up as fewer mangled names and terms in the transcripts, captions and voice searches you rely on. The filing describes the process as repeatable in rounds: gather the suggested data, retrain, and test again to see whether the score improved.

That makes this Microsoft's 22nd filing in voice and speech AI we've tracked since May, a set that also includes training game characters on voice reactions and AI game recaps from play history.

Source: US 2026/0369265 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 12 sheets

12 drawing sheets from US 2026/0369265 A1 · click any drawing to enlarge

Computer tools that pinpoint speech recognition mistakes, drawing sheet 1 from US 2026/0369265 A1
Sheet 1 Computer tools that pinpoint speech recognition mistakes
Testing voice software with real and synthesized speech, drawing sheet 2 from US 2026/0369265 A1
Sheet 2 Testing voice software with real and synthesized speech
Measuring how often sound quality causes speech errors, drawing sheet 3 from US 2026/0369265 A1
Sheet 3 Measuring how often sound quality causes speech errors
Sorting mistakes caused by vocabulary or grammar confusion, drawing sheet 4 from US 2026/0369265 A1
Sheet 4 Sorting mistakes caused by vocabulary or grammar confusion
Suggesting new voice recordings to improve sound recognition, drawing sheet 5 from US 2026/0369265 A1
Sheet 5 Suggesting new voice recordings to improve sound recognition
Recommending fixes for mispronounced words in the dictionary, drawing sheet 6 from US 2026/0369265 A1
Sheet 6 Recommending fixes for mispronounced words in the dictionary
Listing missing words and phrases to improve text models, drawing sheet 7 from US 2026/0369265 A1
Sheet 7 Listing missing words and phrases to improve text models
A scorecard showing which speech components fail most often, drawing sheet 8 from US 2026/0369265 A1
Sheet 8 A scorecard showing which speech components fail most often
Show the other 4 sheets
Pulling web search results to fix frequent transcription mistakes, drawing sheet 9 from US 2026/0369265 A1
Sheet 9 Pulling web search results to fix frequent transcription mistakes
Steps for pinpointing sound recognition errors using computer speech, drawing sheet 10 from US 2026/0369265 A1
Sheet 10 Steps for pinpointing sound recognition errors using computer speech
Finding vocabulary mistakes to choose better training text, drawing sheet 11 from US 2026/0369265 A1
Sheet 11 Finding vocabulary mistakes to choose better training text
Prompting users to add specific data that fixes voice errors, drawing sheet 12 from US 2026/0369265 A1
Sheet 12 Prompting users to add specific data that fixes voice errors
Patent filing page

Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.