Sony Patents an AI Video System That Stops Faint Duplicate Images Before They Spread
When AI tries to predict what the next video frame looks like, moving objects can leave behind faint, smeared ghost images that compound over time. Sony's new patent describes a dedicated cleanup step that catches and kills those ghosts before they corrupt the AI's next guess.
What Sony's ghost-removal video AI actually does
A security camera watches a person walk across a room. The AI processing that footage doesn't just look at each frame in isolation: it uses what it "remembers" from earlier frames to make better predictions about the current one. That memory trick works well, until someone moves in a way that reveals something that was hidden behind them.
When an object moves out of the way, the spot it was covering, called an occlusion region, can carry a faint leftover impression of the object that was just there. AI systems call this a "ghost," and if the AI uses that ghosted memory to predict the next frame, the error multiplies. You end up with blurry smears or doubled edges that get worse with each passing moment.
Sony's patent inserts a dedicated ghost-detection and removal step between the memory and the AI's prediction engine. The system figures out where ghosts are likely hiding based on how objects moved, cleans those spots up, and only then hands the tidied memory to the neural network. The goal is cleaner, more accurate AI video output, frame by frame.
… an occlusion region detection unit configured to detect an occlusion region in the corrected history in which a ghost has a possibility of occurring, based on a motion vector indicating a motion of a subject between frames; …
Translation: This part finds the exact spots where duplicate video artifacts might pop up.
How the system spots and strips occlusion ghosts per frame
The system chains four components together in sequence to handle each incoming video frame.
- Motion compensation unit: Takes the AI's inference result from a past frame and adjusts it to account for how objects have moved since then, producing what the patent calls "corrected history" (essentially a motion-aligned memory of what the scene looked like).
- Occlusion region detection unit: Examines the corrected history using motion vectors (arrows that describe how far and in which direction each object moved between frames) to identify spots where one object has moved away from in front of another, exposing a previously hidden area. These are the zones where ghost artifacts are likely to appear.
- Ghost removal unit: Runs a targeted cleanup pass on just those flagged regions, replacing or suppressing the stale artifact data before it can influence the next prediction.
- Deep neural network (DNN): Receives both the cleaned-up history and the raw current frame image, then infers (predicts and reconstructs) the current frame using accurate context rather than contaminated memory.
The key design choice is separating ghost detection from the neural network itself. Rather than hoping the DNN learns to ignore ghosts on its own, Sony's approach hands the network pre-cleaned inputs. That reduces the burden on the model and, in theory, produces more consistent results when motion is fast or complex.
The ghost removal unit performs ghost removal processing on the occlusion region.
Translation: This step actively cleans up the potential visual glitches before the next frame is built.
What this means for AI-processed video quality
AI-based video processing is increasingly used in everything from streaming compression to security cameras to broadcast production. Any system that uses past frames to improve current-frame quality (a common technique for sharpening, denoising, or super-resolution) inherits the ghost problem. A dedicated pre-cleaning step like this could make those systems more reliable when scenes involve rapid movement or objects passing in front of each other.
For everyday viewers, the payoff would be fewer of those subtle video artifacts where a moving person leaves a brief smear or doubling effect, especially in fast-action content. For engineers building video AI pipelines, the approach offers a cleaner separation of concerns: handle geometry-based errors geometrically, then let the neural network focus on what it's actually good at.
Sony's 14th filing we've tracked in the AI photo editing race since July builds on earlier applications like one that adds missing people and one that fuses invisible light.
Adding a dedicated cleanup stage before the neural network means more moving parts, each with its own failure mode. If the system misjudges how objects are moving between frames, it flags the wrong areas, and the cleanup step strips out image data that was actually fine, swapping one visual glitch for a different one.
The fragility shows up most in exactly the situations you'd want the system to handle: fast, chaotic footage where motion is hardest to track reliably. In calm, predictable scenes, this pipeline likely earns its complexity. In difficult ones, the added step could make things worse.
The underlying logic is still sound. Asking a neural network to clean up geometric artifacts while simultaneously reconstructing fine detail is a harder problem than handing it pre-cleaned input. The trade reads as reasonable, but its value sits almost entirely on how well the motion-tracking step performs upstream.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
18 drawing sheets from US 2026/0289753 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in