Samsung Patents an AI Avatar That Matches Its Face and Body Language to Its Words
Most AI assistants are a voice or a block of text. Samsung's patent describes an avatar that nods while you talk, smiles when its own words are happy, and keeps its body still whenever its mouth is still.
What Samsung's talking AI avatar actually does
Every time a digital character talks, your brain checks whether the mouth, face and body all agree with the words. When they don't, the character looks stiff and a little creepy.
Samsung's patent describes a way to fix that for AI assistants and video-call avatars. The system listens to the speech and reads the words for feelings such as joy, sadness or anger, including how strong each one is. It then moves the avatar's lips, eyes, head and body together, so a happy sentence gets a happy face and a matching gesture, and the body stays still when the mouth does.
The filing says the whole thing is built to run on the device itself, whether that is a phone, a headset, AR glasses or a robot. The avatar can even nod along while you talk to it, so it looks like it is listening.
… providing, using the at least one processing device, the audio features to a facial expression generation model trained to predict at least one of phonemes, phoneme/word boundaries for lip-sync activity, facial blendshape weights for facial expressions, iris movements and eye blink activity for eye movements, and orientation of a neck joint for head movements of an avatar; …
Translation: The system uses audio data to control the avatar's mouth movements, blinks, eye shifts, and head turns.
How the avatar's face and body stay in step
The pipeline takes two inputs: audio and text. Depending on the mode you pick, the audio is your own voice (speech recognition turns it into text), or the text is something you typed (text-to-speech turns it into a voice), or the text is a reply from an on-device large language model (the AI engine behind a chatbot) that is then spoken aloud.
From there the work splits into tracks that share one audio encoder (a model that boils sound down to numbers such as pitch and loudness):
- Face: a model predicts phonemes (the basic speech sounds), where words begin and end, eye blinks, iris movement and head tilt. These become blendshape weights, numbers that say how far to bend each part of a 3D face. The final face blends lip-sync, subtle raw expressions and emotion using an adaptive weighted sum.
- Emotion: a multilingual language model labels the text with an emotion (joy, love, surprise, sadness, fear, anger or neutral) and a strength. A dictionary filter first skips text with no emotional words, which saves time.
- Body: a gesture model uses the audio and emotion to predict where the body's center sits and how each joint is angled, which become animation keyframes (the poses the avatar hits over time).
Body speed fades to zero during pauses in speech, so the body settles when the mouth does. Users can pick head-only, head-and-upper-body, or full-body animation. In the filing's example, a 120 millisecond slice of audio took about 37 milliseconds to process, and the models total under 20 megabytes.
Multi-modal artificial intelligence (AI) generation of synchronized face and full-body animation for a three-dimensional (3D) avatar includes obtaining input audio and text data, extracting audio features, and extracting emotion features using a multilingual language model trained to classify emotion type and corresponding intensity from text.
Translation: An AI program analyzes both voice and text to figure out the emotional tone and how intense it should be.
What this means for AI assistants and AR glasses
If you have used a voice assistant, you know it is usually just a voice or a block of text. This patent aims to give that assistant a face and a body that react to what it says, running on the phone, headset or glasses you already own. The filing names AR glasses specifically, where a full-body assistant could appear in your room as it answers you.
It also covers video calls: your voice drives an avatar of you, and its expressions follow the feelings in your words. The filing points out that viewers are very sensitive to timing mismatches between speech and motion, which is why keeping the face and body in step is the central problem. Running locally also helps the avatar react quickly, as the roughly 37 millisecond example suggests.
Samsung's sixth filing in the AI simulation work we cover since May adds to a set that includes one on avatar body movements and one on simulating atom interactions.
Claim 1 is wider than the title suggests in one way and tighter in another. The wide part: each model only has to predict at least one item from its list, so a product that predicts just mouth shapes on the face side and just joint angles on the body side would still fit the wording.
The tight part is the recipe. The claim requires audio and text together, a language model that rates both the type and the strength of an emotion in the text, separate face and body models, and audio played in step with both animations. An avatar driven by voice alone, with no text-based emotion step, does not match the wording of claim 1.
The headline trick of keeping the body still when the mouth is still shows up only in claim 3. So if granted, claim 1 would reach talking avatars built on this audio-plus-emotion recipe, and the more distinctive timing details sit in the narrower claims that follow. Overall, it is a competent, practical recipe for avatar assistants.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
18 drawing sheets from US 2026/0311411 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in