Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems
Every AI upscaler has to decide how much of the past it can afford to remember. Sony's new patent solves that by compressing each frame's history before passing it forward, keeping the system fast without throwing the context away.
What Sony's frame-history upscaling actually does
A TV upscaling chip is trying to sharpen a blurry, low-resolution video stream in real time, frame by frame. To do a good job, it needs to remember what the last few frames looked like. But holding all of that in memory gets expensive fast, and in a game console or streaming box, you can't just throw unlimited memory at the problem.
Sony's patent describes a way for an AI model to carry a compressed summary of every frame it has already seen, and use that summary to sharpen the current frame more accurately. Instead of storing the full-resolution details of past frames, the system shrinks that history down before passing it along, so the AI still knows what came before without running out of room.
The practical upshot: the upscaler can produce sharper, more consistent output across many frames, especially in scenes with fast movement or lots of detail, without demanding more hardware than a typical game console carries.
… the machine learning model comprises a cumulative feature information output layer having an n th input frame, where n is a natural number equal to or greater than 2, and (n-1) th auxiliary information based on (n-1) th cumulative feature information indicating features of 1 st to (n-1) th input frames …
Translation: The AI model tracks patterns across multiple video frames by building on information gathered from all previous frames.
How the model compresses and passes frame context forward
The patent describes an image upscaling pipeline where a machine learning model processes a sequence of video frames, one at a time, and produces higher-resolution output frames.
The key mechanism is a cumulative feature information output layer. Think of this as the model's rolling memory. Each time the model processes a new frame, it generates a summary of everything it has learned about all the frames it has seen so far, not just the current one. That summary is called cumulative feature information. The summary from frame N-1 is passed in as context when processing frame N, so the model always has a sense of history.
The problem: by the time you have processed many frames, that cumulative summary could itself become very large, potentially as large as the high-resolution output frames. Sony's solution is a pixel-count adjustment step that deliberately shrinks the cumulative feature map to a resolution smaller than the output frame before passing it forward. This is similar to how a journalist summarizes yesterday's news into a short brief rather than reprinting every article.
The result is a system that:
- Upscales low-resolution input frames to high-resolution output frames
- Carries context across frames using a compressed history buffer
- Keeps memory use bounded so the approach can run on fixed hardware like a game console
The processor adjusts the number of pixels that the (n-1) th cumulative feature information comprises so that the number of pixels is smaller than the number of estimated pixels, and acquires the (n-1) th auxiliary information, based on (n-1) th cumulative feature information having the adjusted number of pixels.
Translation: The system shrinks the size of stored data about past frames to save memory while still keeping the details needed for upscaling.
What this means for real-time PlayStation upscaling
For PlayStation players, sharper visuals at lower hardware cost is the direct promise. AI upscaling (generating more pixels than the GPU originally rendered) is already a major part of how consoles compete with high-end PCs, and the quality of that upscaling depends heavily on how well the system tracks motion and detail across frames. A model that carries a compressed history of prior frames should produce fewer artifacts and smoother edges in fast-moving scenes, where frame-to-frame consistency is hardest to maintain.
The memory-efficiency angle is what makes this industrially serious. Upscaling on a PC can lean on large GPU memory buffers; a console has strict limits. Sony's approach, keeping cumulative context alive but compressed, is a practical answer to a real hardware constraint, and it sits alongside a broader wave of interesting tech patents in the AI-video and real-time rendering space that major platform holders are filing as on-device upscaling becomes a competitive battlefield.
AI upscalers that treat each frame in isolation make errors that accumulate visibly, and fixing that by storing full-resolution frame history is too memory-hungry for consumer hardware; that ongoing bind is what Sony is working on here. The compressed-history approach is a proportionate engineering response to a well-defined constraint, incremental and necessary work of the sort that ships in an unannounced firmware update. Whether the chosen compression ratio preserves enough context to matter will depend entirely on implementation details the patent does not pin down.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0237022 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →