Sony Files Patent for an AI That Guesses the Next Game Frame Before It's Drawn
Sony has filed a patent for an AI that looks at the frame you just saw, and your latest button presses, and paints a guess of the next one while the console is still drawing the real thing. The goal is a smoother game without the delay that other frame-boosting tricks add.
What Sony's frame-guessing patent does for games
You're in the middle of a boss fight and the action suddenly stutters because your console can't draw frames fast enough. Frame rates that jump up and down are one of the most noticeable annoyances in modern games.
Sony's filing describes a different trick. While the graphics hardware is busy drawing the next real picture, a trained AI looks at the frame you just saw, and optionally the buttons you just pressed, and guesses what comes next. That guessed frame goes on screen first, so the picture keeps flowing at an even pace.
The filing points out that some existing methods slip a fake frame between two real ones. That means waiting for both real frames to finish, which adds delay. Sony's version makes its guess without waiting for the next real frame to be drawn.
… predicting, by the encoder neural network, image embeddings for the first image frame; using the image embeddings and by a decoder neural network, predicting a next image in the image stream; …
Translation: The AI analyzes one picture to guess what the following picture should look like.
How Sony's encoder and decoder predict a frame
The patent uses an autoencoder, which is a pair of neural networks (software that learns patterns from examples) trained as one unit. The encoder squeezes an image frame into an embedding, a compact list of numbers that captures what is in the picture. The decoder expands that list into a predicted next image.
Training works like this: the weights (the adjustable numbers inside the network) start out random. The encoder gets a frame from a video stream, the decoder predicts the following frame, and the system compares that guess to the actual next frame. The gap is called the loss, a score of how wrong the guess was. The claims tie training to checking that loss against a threshold. No human labels are needed, because the video itself supplies the right answers.
Once trained, the system works in a loop:
- The graphics pipeline draws a real frame.
- The encoder reads it, plus optional motion vectors (arrows showing which way parts of the image are moving) and recent controller input.
- The decoder outputs a synthetic frame straight to the display buffer, shown before the next real frame arrives.
The decoder outputs more color channels than the encoder takes in, which lets it produce several future frames at once. The filing also says it can predict earlier frames, and it describes a simpler non-AI fallback that shifts blocks of pixels along their motion.
Frame generation super sampling may include generating an image frame embedding from a first image frame in an image stream and predicting a synthetic second image frame in the image stream using the image frame embedding of the first image in the image stream.
Translation: The system creates artificial video frames by building on data from the previous picture.
Why Sony's approach could mean less game lag
If this works, you would see steadier frame rates in demanding games without the extra input delay that comes from waiting on two real frames. The filing describes this as a response to "hitching," the sudden slowdowns players notice when a scene gets heavy. It also lets a game keep a fixed pace instead of lowering visual quality to hit one.
The design uses ordinary parts: a processor, memory and a screen, with your controller input feeding the guess. That makes it a software-side idea that could apply to consoles or PCs. The filing gives no measured results, so how much smoother it feels remains an open question.
Sony's 16th filing we've tracked since July in the GPU rendering race builds on one spotting new objects and one sharpening compressed game visuals.
Shipping this needs no new hardware. The system claim asks for a processor, a screen and memory, which describes nearly every game console and PC already on the market. The shortest path to a product is software: train the two networks on game footage, then slot them into a game's graphics routine.
The hard part is the missing evidence. The filing gives no numbers for speed or image quality, and its own example shows the guess missing a detail (a face sticks out its tongue in the real frame but not in the predicted one). A wrong guess flashes on your screen for a split second, so the model has to be right often enough that you never notice.
Until Sony shows that the guesses are fast and accurate in real games, this stays a sound blueprint that is several engineering steps from your living room.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0320740 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in