Sony Patents a System That Notices When Someone Is Missing From a Photo and Adds Them In
You set up the perfect group photo, but the dog wandered off before you hit the shutter. Sony's new patent describes a system that would notice the dog is missing from the frame and add it back in automatically.
What Sony's missing-person photo fix actually does
Ever looked at a family photo and realized the one person you wanted in it just isn't there? Sony is working on a way for a camera or phone to catch that problem and fix it on its own.
The idea works with photos that contain another image inside them, like a picture frame hanging on the wall or a photo album open on a table. The device checks whether a specific person, animal, or object that should be in that inner image is actually there. If it's missing, the device edits the outer photo so the inner image now includes that person or pet.
Think of it as a camera that knows your grandmother belongs in the family portrait on the mantelpiece and makes sure she's there, even if the photo on the wall is old or incomplete. The system handles the detection and the editing without you having to open a separate app or manually paste anything in.
… determine that an entity is not shown in the second image; the entity being a specific person, a specific animal or a specific object; and determine that an image of the entity should be included in the second image; …
Translation: The software figures out who or what is missing from the photo and decides to put them back in.
How the device spots the gap and fills it
The patent describes a pipeline with several distinct steps. First, the device analyzes incoming image data to determine whether a photo (the "first image") contains a second, embedded image inside it, like a framed picture, a screen showing a photo, or a printed snapshot visible in the scene.
Once it finds that inner image, the device checks whether a specific entity (the patent's word for a named person, a particular animal, or a specific object) is absent from it. This implies some kind of recognition system that already knows which entity to look for, presumably from user-defined preferences or a linked profile.
If the entity is confirmed missing and the system determines it should be present, the device then modifies the outer photo so that the inner image now contains the entity. In plain terms: it edits the picture-within-a-picture to fill in who or what is supposed to be there.
- Detect an embedded image inside a larger photo
- Identify a specific person, animal, or object that should appear in that embedded image
- Confirm the entity is absent
- Synthesize and insert that entity into the embedded image
… modify the first image such that a modified first image of the second image includes the image of the entity.
Translation: The system alters the original picture to insert the missing person, animal, or object.
What this means for cameras and family photos
For everyday photography, this would mean your camera could catch a gap you never noticed and fix it before you ever review the shot. Family portraits, wedding albums, memorial photos displayed in a room, any scene where one image sits inside another becomes a candidate for automatic correction.
The practical value depends heavily on how well the recognition and generation layers perform. If the system can accurately identify who belongs in an old framed photo and produce a natural-looking result, that is a meaningful feature for consumer cameras and smartphones. Sony's interest in computational photography shows up in a run of filings around image processing, and this one pushes into generative editing territory that sits alongside tools already appearing in phone software from several companies.
Sony's 13th filing we've tracked in the AI photo editing race since July builds on earlier applications like its invisible light repair patent and one that learns from your feedback.
Claim 1 is written broadly. It covers any electronic device with circuitry that can detect an embedded image, identify a missing entity, decide that entity should be present, and then modify the outer image accordingly. There is no restriction to cameras, to any particular generation method, or even to photographs of physical frames. A tablet showing a photo album on screen would qualify, as would a TV displaying a slideshow.
That breadth is a double-edged thing. It gives Sony wide coverage if the claim survives examination, potentially reaching across consumer cameras, phones, and display devices. But broad claims also attract the most scrutiny from examiners, and the prior art in AI photo editing is dense at this point.
The concept itself is specific enough to be interesting. Detecting what is missing from a picture-within-a-picture, rather than from the main scene, is a narrower problem than general photo completion. Whether that specificity is enough to make the claim patentable, given how much generative image editing has been published recently, is the real open question.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0278875 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →