Nvidia Files Patent to Make Animated Characters Change Mood Mid-Sentence
Most animated characters wear one mood per line of dialogue. Nvidia's filing describes an AI that tracks how emotion shifts through a sentence, trained with a second AI acting as the judge.
How Nvidia's AI reads mood shifts in speech
Every time a video game character talks to you, its face has to decide how to feel. Most systems pick a single emotion for the whole line, so a character who starts out cheerful and ends up furious keeps one expression the entire time.
Nvidia's filing describes a way to listen to speech and work out a sequence of emotions instead: happy for the first few words, say, then angry for the rest. The character's face can then shift along with the voice.
To teach the system what feels right, Nvidia describes having people watch pairs of animated clips of the same speech, each with a different emotion timeline, and pick the more believable one. A second AI learns from those choices and acts as a judge, scoring the first AI's guesses while it trains. The whole thing can run on a remote server that a game or app talks to over the internet.
… determining, by the one or more servers using at least a first machine learning model processing the audio data, a sequence of emotional states corresponding to the speech …
Translation: The server uses artificial intelligence to figure out how the speaker's emotions change over time.
How a second AI judges the first one's guesses
The core of the filing is a service. A remote device (a game, an app, a phone) sends a clip of speech to a server through an API, which is a standard doorway that lets one piece of software send requests to another. The server runs an AI model that returns a sequence of emotional states for the clip and sends it back.
The interesting part is the training. A first model guesses an emotion timeline for some speech. A second model, a kind of judge, looks at the audio plus that timeline and gives it a score for how well it matches the speech. The first model is then adjusted based on the score. The judge itself was trained on human picks between two animated clips of the same speech.
The filing also describes:
- Emotions such as anger, disgust, fear, joy, sad and neutral, tied to moments (keyframes) in the audio.
- A third, earlier copy of the model that keeps the first from drifting too far off course.
- Training until a chosen share of scores, such as 90%, clears a threshold.
On the receiving end, the claims cover a device that animates a character's face to follow the emotion timeline, or generates spoken audio with a matching emotional tone.
… the second machine learning model(s) may be trained to determine the scores using audio data representing speech, sequences of emotional states associated with the speech, and indications of which sequences of emotional states better represent the speech as compared to other sequences of emotional states.
Translation: A second AI system grades how accurately the first system tracked the shifts in emotion.
What this means for talking game characters
If you play games or chat with an AI avatar, the practical payoff is a character whose face keeps up with its voice. Today a line can sound sarcastic and then turn sincere while the expression stays frozen. Tracking emotion across a sentence is a step toward characters that look like they mean what they say.
The filing is also built as a service: audio goes up, emotion data comes back. That would let developers add this to a game or app without building the emotion model themselves. The description lists games, communication apps, and in-car systems as places it could be used, and it says the same method could be applied to gestures and other movements that go with speech.
That makes this Nvidia's 27th filing we've tracked since May in our text-to-3D characters work, building on one adding realistic surfaces and one fixing mouth glitches.
Claim 1 is wider than the headline suggests. It never mentions animation or characters. It covers a server that takes in audio over an API, works out a series of emotions for it, and sends the result back to the device that asked.
The one narrowing piece sits in the middle: the emotion model must have been trained using scores from a second model that judges its guesses. A service that tags a whole clip with one mood does not match what the claim describes.
Faces show up only in later claims, which add animated characters and spoken replies with an emotional tone. If granted, the main claim would cover a cloud emotion-reading service for any app that handles voice, and that reach matters more to developers than the talking-character demo does.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0341272 A1 · click any drawing to enlarge
Show the other 4 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in