Sony Files Patent to Show Mood and Energy in Video Call Captions
Sony wants captions on a video call to show how people feel, not just what they say. Louder voices make bigger bubbles, happy tones turn warm orange, and a dying conversation fades to dark.
What Sony's mood-reading caption bubbles do
Ever sat through a video meeting and had no idea whether anyone liked your idea? Without a room to read, you miss the sighs, the smiles, and the moment the energy dropped.
Sony's filing tackles that gap. A server listens to everyone's microphone and watches each camera feed, then turns what people say into floating circles of text. The circles change with the conversation: louder speech makes a bigger circle, a cheerful tone makes it warmer in color, and a gloomy one makes it cooler.
The filing says this could help people who are hard of hearing, since they can read the words but miss the emotion. It also lets you switch each visual effect on or off and dial its strength up or down.
a data analysis unit that analyzes received data from a user terminal carrying out a conversation via a communication network; and a display data control unit that controls display data having an utterance-text display region including text corresponding to a user utterance to be provided to the user terminal, on a basis of an analysis result of the data analysis unit …
Translation: The system reads chat data and changes how captions look based on what it finds.
How Sony's server turns voice into bubble size
The system has two main parts. A data analysis unit takes in voice and camera data from each participant's device. A display data control unit then decides how each caption bubble should look, based on that analysis.
The filing lists thirteen things the bubbles can show. Here are some of them:
- Volume: louder speech makes a bigger circle.
- Tone: a cheerful voice gets a bright or warm color, a depressed one gets a dark or cold color, and effects like a glow or a cloud can appear around it.
- Keywords: key words in a sentence are enlarged or highlighted in a color such as red.
- Replies: a positive reply drifts toward the comment it answers and takes a similar color, while a negative reply moves away and takes a different color.
- Group mood: when the chat is lively, the background brightens and bubbles bounce. When it fades, the background darkens.
The server also reads facial expressions, nods and head shakes. In one variation, it uses body signals like pulse, temperature and where you are looking, from sensors you wear. Bubbles drift across the screen and fade as they age, and opinion groups (say, dog people versus cat people) cluster apart.
A display data control unit controls the size, position, color, or a movement of the utterance-text display region or at least one of output images around the utterance-text display region, and generates display data visualizing at least one of a user state and a conversation state of the user carrying out a conversation.
Translation: It alters text size, color, movement, or background visuals to show how participants feel.
Why mood-tinted captions matter for remote meetings
If your work or classes happen over video, this could change how a call feels. A quiet participant's reaction would show up on screen, and a meeting that has gone flat would look flat. For hearing-impaired users, the extra cues could fill in what a plain transcript leaves out. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments.
The filing also raises a privacy question, because the tool reads your face, tone and possibly your pulse. It does let each user turn individual effects off, which is an important part of the design. The filing does not name a Sony product, so treat it as an idea on paper.
Sony's 640th filing in our Sony coverage since May follows work like turning game sounds into haptics and faster graphics compression.
The trade here is accuracy against attention. Every effect depends on software guessing your mood from a voice or a face, and those guesses will sometimes be wrong. A bubble that turns gloomy blue because you spoke softly could misrepresent you in front of your boss.
There is also a clutter cost. Bubbles that grow, shake, flash, drift and change the background color could pull your eyes away from the people you are talking to. The on/off and strength sliders in the filing help, and they are probably the most important part of the design.
On balance, the trade looks worth making for readers who cannot hear the call, where the gain is large. For everyone else, it depends on how restrained the final design turns out to be.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
45 drawing sheets from US 2026/0329506 A1 · click any drawing to enlarge
Show the other 37 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in