Nvidia Files Patent to Stop Transcription Software From Hearing 'NVIDIA' as 'a Video'
Speech software loves to turn a company name into 'a video' and a conference acronym into 'jet ski.' Nvidia's filing describes handing the software a keyword cheat sheet, then training it to doubt that cheat sheet when the audio disagrees.
What Nvidia's keyword-hint transcription does
You're recording a talk about Nvidia's GTC conference, and the transcript comes back with 'a video' where it should say NVIDIA and 'jet ski' where it should say GTC. Speech-to-text software tends to stumble on names, acronyms and technical terms, which is exactly where you need it to be right.
This Nvidia filing describes a way to give the software a cheat sheet. Along with your audio, it receives a short list of keywords likely to come up, plus a plain instruction such as 'transcribe this' or 'translate this into Spanish.' One setup handles both jobs.
The clever part is how it gets taught. During training, the keyword list deliberately includes some words that have nothing to do with the recording. That teaches the software to lean on a hint only when the audio backs it up, so a wrong suggestion doesn't end up in your transcript.
a first portion comprising a training audio input in a first language, a second portion comprising a text context associated with the training audio input, and a third portion identifying a type of a speech-to-text (S2T) conversion of the training audio input to be generated using a machine learning model (MLM); …
Translation: The system feeds audio, surrounding text clues, and conversion instructions into the AI all at once.
How the hints and instruction reach the model
The claims cover a training method for a speech-to-text model built from two parts. An audio model listens to the recording and produces an intermediate output (a string of tokens, which are number codes a language model can read). A language model (the same kind of software behind chatbots) then reads those tokens together with two text pieces.
The training input has three parts:
- The audio, in a first language.
- A text context, usually a list of keywords tied to the audio.
- An instruction naming the job: transcribe in the same language, or translate into a second one.
The model's output is compared with a correct answer (the ground truth), and that error is used to train the audio model. The audio side can include a speech network and an adapter that converts its output into tokens, using conformer layers (a design mixing two common neural network types) and a step that lowers the frame rate.
Keywords are picked at random, partly from the correct transcript and partly from a store of other training examples. Take a finance clip: the list might hold 'ETF' and 'S&P 500' alongside 'diplomacy' and 'unemployment,' which never appear in the speech. The filing also describes keeping the language model frozen and training only a small add-on, which makes training faster and cheaper.
Disclosed are apparatuses, systems, and techniques that implement training and deployment of speech-augmented language models for efficient capturing and processing of speech inputs.
Translation: Nvidia invented a method to train AI systems so they better understand spoken words by combining audio and text.
Why jargon-heavy transcripts could get cleaner
If this works as described, the payoff is cleaner transcripts for conversations full of specialized words: live captions, meeting notes, clinical notetaking, and voice assistants in cars or kiosks, all of which the filing lists as settings. Instead of retraining a model for every industry, you could hand it a vocabulary list for the topic at hand.
The single-model design matters too. The same system can transcribe or translate depending on one line of instruction, rather than needing separate software for each job. The filing also describes the system working out the subject of a recording on its own and pulling a stored keyword list for it, so you might not have to type hints at all.
This is the 23rd Nvidia filing we've tracked in voice & speech AI since May, a topic that also covers one to fix mouth glitching and one routing audio to processors.
A hint can steer you wrong. If the keyword list says 'GTC' and the speaker said something else, a model that trusts the list too much will write down the wrong word with full confidence. Nvidia's answer is to pad the training lists with unrelated words, which forces the model to check the audio first.
That fix has its own price. A model taught to doubt hints may now and then ignore a correct one, and someone still has to supply the list. That means a person typing it, or an extra first pass where the language model guesses the topic, which adds processing time.
The trade reads as worth it. Misheard names and acronyms are the errors that make a transcript hard to trust, and a bit of extra delay or occasional over-caution is a small cost next to that.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0341280 A1 · click any drawing to enlarge
Show the other 2 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in