IBM Patents an AI That Reads Your Emotions on Video Calls and Reacts in Real Time
IBM has filed a patent for a system that watches and listens to everyone on a video call, figures out how they're feeling, and then automatically adds emotional cues into the video stream, without anyone pressing a reaction button.
What IBM's emotion-detection system actually does on calls
Imagine you're sitting in a long video meeting, and someone just delivered news that landed badly, but they can't tell because everyone's faces are small thumbnails on a screen. IBM's new patent is designed to fix exactly that gap.
The system uses AI to monitor the audio and video coming from each participant's device during a live call. It then figures out each person's emotional state, think frustrated, excited, confused, or engaged, and automatically overlays a corresponding visual element (like an emoji, icon, or color signal) onto that person's video feed.
The key part is that you don't have to do anything. There's no clicking a thumbs-up or a heart reaction. The system picks up on your tone of voice, your facial expressions, or other cues, makes its own call about how you're feeling, and inserts that feedback into what other people on the call can see.
How the two-stage AI model reads and labels feelings
The patent describes a two-stage AI pipeline that runs during a live call.
Stage one collects raw communication data from each participant's device, this could include video frames, audio signals, or text from chat, and runs it through a first set of AI models to pull out what the patent calls "key features indicative of an emotional state." Think of this as the system identifying relevant signals: the pitch in your voice, the expression on your face, or your word choices.
Stage two takes those extracted features and feeds them into a second set of AI models that classify the signals into a predefined emotional category from a fixed list (the patent calls these "emotional feedback classifications").
Once classified, the system looks up a matching emotional feedback element from a library, essentially a catalog of visual or interactive cues tied to each emotion category. That element is then inserted directly into the participant's data stream, meaning other people on the call see it as part of the live video.
The whole process happens continuously and automatically, in real time, across all participants on the call.
What this means for the future of workplace video calls
Video calls have a well-known problem: it's hard to read the room. Body language is compressed, reactions are delayed, and most people don't bother clicking the emoji button mid-conversation. A system that automatically surfaces emotional feedback could make remote meetings feel more like being in the same room, where you naturally pick up on how people are responding.
For IBM, which sells a range of enterprise collaboration and AI tools, this fits into a broader push to make AI a layer sitting underneath everyday workplace software. The practical concern worth watching is consent and accuracy: if the AI misreads your furrowed brow as anger when you're just squinting at a bright window, that feedback could create more confusion than it resolves.
This is one of those patents that sounds either genuinely useful or mildly unsettling depending on your tolerance for AI reading your face at work. The core idea, passively surfacing emotional context that people miss on video calls, solves a real problem. Whether anyone will trust it, or want it running on their webcam feed without explicit opt-in, is a much bigger question that the patent doesn't address.
The drawings
3 drawing sheets from US 2026/0197358 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.