Sony Patent Splits Shadow and True Color Data From Any Photograph
Every photo you've ever taken is a mix of two things: the actual color of an object and the shadows cast by whatever light happened to be in the room. Sony is patenting a way to pull those two things apart.
What Sony's shadow-stripping image system actually does
Ever looked at a photo and wondered why the colors look so different from what you saw in real life? A big part of the answer is shadows. Light hits objects at angles, creates shading, and the camera captures all of it blended together into one image.
Sony's new patent describes a system that tries to separate those two layers. It takes your main photo (shot under whatever lighting happened to be there) and pairs it with a second scan of the same scene using a controlled light source at a different wavelength, think of it like a structured flash. It also takes a depth map of the scene. Together, those inputs let the system estimate what the subject's surface would look like if there were no shadows at all, and then separately calculate where the shadows actually are.
The result is two clean outputs: a "true color" image showing what the subject actually looks like without lighting tricks, and a shadow-only image. That kind of separation is useful for 3D scanning, medical imaging, machine vision on factory floors, or anywhere a machine needs to understand a surface's actual color rather than the lighting conditions around it.
… a first wavelength diffuse reflectance and shade estimation unit that estimates a first wavelength diffuse reflectance image and a first wavelength shaded image that are a diffuse reflectance image and a shaded image of the subject …
Translation: The system uses a specialized component to separate a photo into its true colors and its shadows.
How the two-wavelength scan isolates diffuse reflectance
The system takes three inputs at once: a first wavelength band image (the main photo, captured under ambient or unknown lighting), a second wavelength band image (a scan of the same scene under a controlled, known light source at a different wavelength, such as near-infrared), and a distance image (a depth map showing how far each part of the scene is from the camera).
From the controlled-light scan and the depth map, the system estimates what's called a diffuse reflectance image for that second wavelength. Diffuse reflectance is the base color a surface reflects in all directions equally, stripped of highlights and shading. Once the system knows the surface's reflectance properties from the controlled scan, it can use that as a reference to work backwards through the main ambient-light photo.
Using that reference, a second processing unit estimates two things for the main photo's wavelength band:
- A diffuse reflectance image: what the subject's surface actually looks like, independent of lighting
- A shaded image: the lighting and shadow layer on top of that surface
The key insight is that shooting the same scene in two different wavelength bands, one controlled and one ambient, gives the algorithm enough information to separate what's "painted on" the surface from what's "cast" onto it by the light environment.
The technology of the present disclosure can be applied to, for example, an image processing device or the like that separates a visible band image into a diffuse reflectance image and a shaded image.
Translation: This technology can be used in devices to automatically pull shadows out of standard color photographs.
What this means for cameras and computer vision systems
For consumer cameras, this kind of processing could allow computational photography to produce more accurate color grading or better subject separation in mixed or tricky lighting. For industrial and medical imaging, the ability to strip shadows automatically without needing a controlled studio setup is a bigger deal: machines inspecting parts on an assembly line or cameras scanning skin lesions need to see what's actually there, not what the overhead fluorescent lighting is doing to it.
Sony is one of the world's largest image sensor manufacturers, supplying components to companies across mobile, automotive, and medical markets, so patents in image processing often have a longer reach than they first appear. This filing sits alongside a broader wave of new Big Tech patents in computational imaging, where companies race to extract more signal from camera hardware without changing the physical optics.
The patent relies on having two synchronized imaging channels at different wavelengths, which means it needs specific hardware: a camera rig or sensor array that can capture both a controlled-wavelength scan and a standard image at the same moment. That's a meaningful barrier. Software-only deployment on existing smartphone cameras isn't the obvious first step here. The most direct path to a product is probably in Sony's own industrial or medical imaging lines, where controlled illumination is already standard practice and adding a second wavelength channel is an engineering refinement rather than a redesign. A consumer camera application would require either a new sensor design or a dual-camera approach with synchronized illumination, which is feasible but not trivial.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
18 drawing sheets from US 2026/0237087 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →