Get the newsletter
Sony · AI / ML · Filed Mar 30, 2026 · USPTO published Aug 13, 2026 · Aug 13 release

Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems

Every AI upscaler has to decide how much of the past it can afford to remember. Sony's new patent solves that by compressing each frame's history before passing it forward, keeping the system fast without throwing the context away.

Step-by-step progression of a video frame being upscaled from low resolution to high resolution using machine learning and motion compensation. Drawing from patent filing US 2026/0237022 A1.
Step-by-step progression of a video frame being upscaled from low resolution to high resolution using machine learning and motion compensation.All 12 drawings ↓
Publication number
US 2026/0237022 A1(US20260237022A1)
Applicant
Sony Interactive Entertainment Inc.
Filed · Published
Mar 30, 2026 · Aug 13, 2026
Inventors
Masato Mitsutake
Status when we published
Waiting for an examiner
Document
12 claims
US classification
463/31
Parent application
Continuation of PCT/JP2024/033479 (filed Sep 19, 2024)(PCTJP2024033479)

What Sony's frame-history upscaling actually does

A TV upscaling chip is trying to sharpen a blurry, low-resolution video stream in real time, frame by frame. To do a good job, it needs to remember what the last few frames looked like. But holding all of that in memory gets expensive fast, and in a game console or streaming box, you can't just throw unlimited memory at the problem.

Sony's patent describes a way for an AI model to carry a compressed summary of every frame it has already seen, and use that summary to sharpen the current frame more accurately. Instead of storing the full-resolution details of past frames, the system shrinks that history down before passing it along, so the AI still knows what came before without running out of room.

The practical upshot: the upscaler can produce sharper, more consistent output across many frames, especially in scenes with fast movement or lots of detail, without demanding more hardware than a typical game console carries.

From the filing · CLAIM 1
… the machine learning model comprises a cumulative feature information output layer having an n th input frame, where n is a natural number equal to or greater than 2, and (n-1) th auxiliary information based on (n-1) th cumulative feature information indicating features of 1 st to (n-1) th input frames …

Translation: The AI model tracks patterns across multiple video frames by building on information gathered from all previous frames.

How the model compresses and passes frame context forward

The patent describes an image upscaling pipeline where a machine learning model processes a sequence of video frames, one at a time, and produces higher-resolution output frames.

The key mechanism is a cumulative feature information output layer. Think of this as the model's rolling memory. Each time the model processes a new frame, it generates a summary of everything it has learned about all the frames it has seen so far, not just the current one. That summary is called cumulative feature information. The summary from frame N-1 is passed in as context when processing frame N, so the model always has a sense of history.

The problem: by the time you have processed many frames, that cumulative summary could itself become very large, potentially as large as the high-resolution output frames. Sony's solution is a pixel-count adjustment step that deliberately shrinks the cumulative feature map to a resolution smaller than the output frame before passing it forward. This is similar to how a journalist summarizes yesterday's news into a short brief rather than reprinting every article.

The result is a system that:

  • Upscales low-resolution input frames to high-resolution output frames
  • Carries context across frames using a compressed history buffer
  • Keeps memory use bounded so the approach can run on fixed hardware like a game console
From the filing · THE ABSTRACT
The processor adjusts the number of pixels that the (n-1) th cumulative feature information comprises so that the number of pixels is smaller than the number of estimated pixels, and acquires the (n-1) th auxiliary information, based on (n-1) th cumulative feature information having the adjusted number of pixels.

Translation: The system shrinks the size of stored data about past frames to save memory while still keeping the details needed for upscaling.

What this means for real-time PlayStation upscaling

For PlayStation players, sharper visuals at lower hardware cost is the direct promise. AI upscaling (generating more pixels than the GPU originally rendered) is already a major part of how consoles compete with high-end PCs, and the quality of that upscaling depends heavily on how well the system tracks motion and detail across frames. A model that carries a compressed history of prior frames should produce fewer artifacts and smoother edges in fast-moving scenes, where frame-to-frame consistency is hardest to maintain.

The memory-efficiency angle is what makes this industrially serious. Upscaling on a PC can lean on large GPU memory buffers; a console has strict limits. Sony's approach, keeping cumulative context alive but compressed, is a practical answer to a real hardware constraint, and it sits alongside a broader wave of interesting tech patents in the AI-video and real-time rendering space that major platform holders are filing as on-device upscaling becomes a competitive battlefield.

Source: US 2026/0237022 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 12 sheets

12 drawing sheets from US 2026/0237022 A1 · click any drawing to enlarge

Hardware components of the processing system, drawing sheet 1 from US 2026/0237022 A1
Sheet 1 Hardware components of the processing system
Machine learning model architecture for upscaling, drawing sheet 2 from US 2026/0237022 A1
Sheet 2 Machine learning model architecture for upscaling
Frame-by-frame video upscaling process, drawing sheet 3 from US 2026/0237022 A1
Sheet 3 Frame-by-frame video upscaling process
Functional block diagram of the upscaling system, drawing sheet 4 from US 2026/0237022 A1
Sheet 4 Functional block diagram of the upscaling system
Virtual camera capturing 3D objects into frames, drawing sheet 5 from US 2026/0237022 A1
Sheet 5 Virtual camera capturing 3D objects into frames
Pixel interpolation grid, drawing sheet 6 from US 2026/0237022 A1
Sheet 6 Pixel interpolation grid
Horizontal pixel expansion pattern, drawing sheet 7 from US 2026/0237022 A1
Sheet 7 Horizontal pixel expansion pattern
Multi-directional pixel expansion pattern, drawing sheet 8 from US 2026/0237022 A1
Sheet 8 Multi-directional pixel expansion pattern
Show the other 4 sheets
Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems patent drawing, sheet 9
Sheet 9
Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems patent drawing, sheet 10
Sheet 10
Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems patent drawing, sheet 11
Sheet 11
Sony Patent Targets Memory Efficiency in AI-Powered Video Upscaling Systems patent drawing, sheet 12
Sheet 12
Patent filing page

More in AI image & video

All 223 AI image & video patents →

Watchlists this filing belongs to

All watchlists →