Sony Patents a System That Scores What Viewers Actually Watch On Screen
Sony has filed a patent for a system that uses cameras pointed at an audience to track where people's eyes land on a video, then produces a numerical score for how much attention each on-screen object actually received.
How Sony's gaze-tracking engagement score works
Ever wondered whether viewers are actually looking at the logo in your ad, or just zoning out at the background? That's the exact problem Sony's new patent is trying to solve.
The system works by watching the audience while they watch the video. A camera captures images of the viewers, an AI model figures out where each person's gaze is pointed, and that gaze data gets mapped back onto the video frame. Every on-screen object, a product, a face, a text overlay, gets an engagement score based on how many people looked at it and for how long.
The result is a dashboard-style output that shows which parts of a video actually grabbed attention and which parts went unnoticed. For anyone producing video content or placing ads inside it, that's a measurably different kind of feedback than view counts or skip rates.
… receive a set of images of an audience viewing the received media content; estimate, based on application of a second machine learning model on the set of images, gaze data associated with the audience; map the estimated gaze data to the set of objects detected in the plurality of video frames …
Translation: The system uses cameras to watch you while you watch TV, tracking exactly where your eyes are looking on the screen.
How two AI models turn eye data into per-object scores
The patent describes an electronic device running two separate neural network (machine learning) models in sequence.
The first model processes the video itself, scanning each frame to identify and locate objects: a face, a brand logo, a product being demonstrated, any discrete element the frame contains. The second model processes images of the audience captured while they watch, estimating gaze coordinates (the x/y position on screen each viewer appears to be looking at).
Once both models have done their work, the system divides each video frame into a grid of segments (smaller zones of the image) and maps the gaze coordinates into those segments. Because it knows which objects occupy which segments, it can calculate how much collective viewer attention landed on each object over time. That produces a per-object engagement score.
The patent also references a content playability factor, a condition the system checks before deciding which video frames to analyze, though the filing does not fully specify what that condition entails. The final scores are rendered on a display device, suggesting a monitoring or analytics interface rather than something end-viewers would see.
The electronic device determines an engagement score for each object based on the mapping of the gaze co-ordinates and renders the engagement score on a display device.
Translation: The device calculates a score for every item on screen to show how much attention it received from the viewer.
What this means for advertisers and content producers
For advertisers and broadcasters, this kind of data would answer a question that click-through rates and completion percentages cannot: did anyone actually look at the thing we paid to place on screen? A brand spending money on a product placement in a streaming series, or on an in-video overlay ad, currently has very little proof that viewers' eyes ever went there. An engagement score per object changes that calculus entirely.
The practical path to a shipping product requires hardware (audience-facing cameras in a controlled setting like a focus-group room, a smart TV, or a cinema) plus the consent and privacy infrastructure to collect gaze data legally, so this is not something that appears in a firmware update next quarter. Still, the underlying computer-vision pieces are mature, and Sony operates across televisions, cameras, and entertainment, which puts multiple parts of the required stack already inside one company. Readers tracking new Big Tech patents in ad tech and audience measurement will recognize this as part of a broader push to make attention, not just playback, the currency of video analytics.
The ship path for this patent is long and multi-step: you need audience cameras, real-time gaze estimation, object detection on the video side, and a legal framework for collecting viewer biometric data, all running together. None of those pieces is missing from Sony's existing portfolio, but integrating them into a deployable product takes more than software. The most realistic near-term form is a controlled research or focus-group tool, with any consumer application far behind it. That makes this a credible R&D filing whose value lies in locking down the method before competitors formalize similar systems.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0237091 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →