Sony Patents a Fix for AI Image Enhancement That Breaks Down in Extreme Lighting
AI upscaling is great at making low-resolution images look sharper, but it tends to fall apart when a scene is unusually bright or unusually dark. Sony has filed a patent describing a fix: briefly recolor a frame to look more like the images the AI was trained on, run it through the upscaler, then recolor it back.
What Sony's brightness-aware upscaling actually does
A video game renders a sun-drenched desert, then cuts to a pitch-black dungeon. Both look very different from the average training image an AI upscaler was built on, so the upscaler guesses wrong and the result looks blurry or wrong.
Sony's patent describes a system that, before handing a frame to an AI upscaler, measures how bright or dark that frame is and nudges its color values toward what the AI expects. The AI upscales the adjusted frame, then a second step nudges everything back to the original brightness level. The user sees a sharper image without the washed-out or crushed look that can happen when an AI gets input that looks nothing like its training data.
The whole process is designed to work in real time. The adjustment going in and the reversal coming out are mirror operations, so no brightness information is lost and the final image reflects exactly what the game originally intended.
… performing, for the first processed frame, distribution transformation that transforms the luminance distribution of the pixels contained in the first processed frame into luminance distribution close to that of the pixels contained in a learning input frame on a basis of the first parameter to generate a second processed frame …
Translation: Adjusts the image brightness to match what the AI model expects to see before trying to upscale it.
How the luminance transform wraps around the AI upscaler
The system works in four stages, wrapping around a standard AI resolution-upscaling model (called the resolution conversion section in the patent).
- Measure brightness: The system analyzes the luminance distribution (roughly, the spread of dark-to-light pixel values) of the incoming low-resolution frame and calculates a parameter that summarizes how bright or dark it is overall.
- Distribution transformation: Using that brightness parameter, the system reshapes the frame's pixel values so they more closely resemble the training frames the AI upscaler was built on. Think of it as color-grading the image into the AI's comfort zone. This produces a second, adjusted frame.
- AI upscaling: That adjusted frame is fed to the AI upscaler, which generates a higher-resolution version. Because the input now looks like the AI's training data, the AI makes better predictions.
- Inverse transformation: The upscaled frame is then color-graded in reverse, using the same brightness parameter in reverse, to restore the original scene's intended look at the higher resolution.
The key constraint the patent enforces is that all pixel values involved are in linear light space (meaning the numbers are proportional to actual physical brightness, not gamma-encoded as most consumer screens expect). That keeps the math for the forward and reverse transforms consistent and lossless.
… inputs the second processed frame to a resolution conversion section to generate a third processed frame having an estimated number of pixels each of which has a pixel value having a linear relation with luminance …
Translation: Feeds the adjusted image into the AI upscaler to add more pixels and increase the resolution.
What this means for PlayStation graphics quality
AI upscaling, whether through Sony's own PlayStation upscaling tech or rivals like Nvidia's DLSS and AMD's FSR, is increasingly how games hit high resolutions without needing proportionally more GPU power. The practical weakness is that all of these systems are trained on a finite library of reference images, and any frame that looks very different from that library (extremely high dynamic range, very dark interiors, bloom-heavy outdoor shots) can produce artifacts.
If Sony bakes this correction stage into its upscaling pipeline, it could mean more consistent image quality across a wider variety of game environments, including the edge cases (explosions, night levels, HDR sunsets) where current upscalers visibly struggle. For players, the direct benefit is fewer moments where the background looks oddly soft or smeared compared to what the raw rendering would have produced.
Sony's 15th filing we've tracked in the AI photo editing race since July builds on work seen in the ghost-frame fix and the missing-person fill-in: using AI to correct what cameras get wrong.
The core trade Sony is making here is speed over flexibility: instead of teaching the AI to handle dark or bright footage directly, the system mathematically massages the image before the AI sees it, then reverses that massage afterward. That round-trip is cheap to run, but any imperfection in the reversal step can introduce subtle color shifts, especially at the brightest and darkest edges of an image where the math is least forgiving.
The deeper cost is that this fix is narrow by design. It addresses one known weakness, brightness handling, and leaves everything else, motion blur, fine text, reflections, entirely to whatever the underlying AI can already do.
That narrowness might actually be the point. A targeted, low-cost patch for a documented real-world problem is exactly what a consumer electronics company ships into a product pipeline. Whether viewers ever notice the difference is the honest open question.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0303982 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in