Sony Patents AI That Sharpens Video Detail Without Being Fooled by Shifting Colors
AI video upscaling usually assumes that pixels change because something moved. Sony's new patent tackles a trickier problem: what happens when pixels change color for reasons that have nothing to do with movement, like flickering lights or shimmering water?

- Publication number
- US 2026/0179183 A1(US20260179183A1)
- Applicant
- Sony Group Corporation
- Filed · Published
- Dec 29, 2025 · Jun 25, 2026
- Inventors
- Kenichiro Yokota, Hisashi Kobiki, Shoichi Ikenoue
- Status when we published
- Waiting for an examiner
- Document
- 21 claims
- US classification
- 382/299
- Parent application
- Continuation of PCT/JP2024/022662 (filed Jun 21, 2024)(PCTJP2024022662)
What Sony's texture-aware video upscaling actually does
Imagine watching a video of a campfire or a neon sign reflected in wet pavement. The texture of those surfaces is constantly shifting color, even when nothing is moving. Most AI systems that sharpen or upscale video footage get confused by this, because they're trained to track movement and use it to fill in missing detail.
Sony's patent describes a system that specifically spots those "color change" pixels, the ones that are shifting due to texture effects rather than actual motion, and tells the AI to treat them differently. Instead of letting those unstable pixels corrupt its frame-to-frame memory, the system replaces them with a neutral placeholder before passing information forward.
The result is an upscaling process that can still make low-resolution video look sharper, even when the scene contains objects like fire, water, or fabric whose surface appearance is constantly in flux. It's a targeted fix for one of the more stubborn blind spots in current video AI.
How the model flags and neutralizes color-change pixels
The system works by processing a sequence of video frames through several stages. First, each low-resolution input frame is enlarged into an intermediate frame with more pixels. Those intermediate frames are then fed into a machine learning model that outputs a final high-resolution estimated frame.
The key innovation is in how the model carries memory between frames. It maintains cumulative feature information (a running summary of what it has seen across all previous frames) to help it reconstruct fine detail. But if a pixel is changing color because of texture effects rather than movement, including it in that running summary would pollute future frames with bad data.
To prevent this, the system identifies color change pixels by comparing texture information across frames and flagging pixels whose color shifts regardless of object motion. Before that frame's data gets folded into the cumulative memory, those flagged pixels are replaced with a fixed placeholder value.
The model is trained specifically on data that includes these replacements, so it learns to produce sharp estimated frames even when portions of the input are intentionally blanked out. The training pipeline mirrors the inference pipeline, which helps the model generalize to real-world footage with unpredictable surface textures.
What this means for AI upscaling in cameras and TVs
For Sony, which makes cameras, TVs, and image sensors, this kind of fix matters across multiple product lines. AI upscaling is already a selling point on high-end televisions, and it's increasingly used in professional video production. The gap this patent addresses is real: footage of fire, water, foliage, or any reflective surface is notoriously difficult for frame-interpolation and super-resolution systems.
For you as a viewer or videographer, the practical upside would be fewer artifacts and sharper detail in exactly the scenes that currently look the worst after AI processing. Whether this ends up in a TV firmware update, a camera processor, or a post-production software tool, it targets a concrete limitation rather than just a marginal quality improvement.
Source: US 2026/0179183 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
