Sony Files Patent for Turning Gameplay Video Into a Single Image to Catch Bugs
Sony Interactive Entertainment wants to check a whole stretch of gameplay by looking at one still picture built from a thin slice of every frame. Straight lines and slanted lines in that picture reveal whether your button press did what it should.
What Sony's one-image video check does
Every time a game gets updated, someone has to press the same buttons again to make sure nothing broke. Does the menu open? Does the character walk forward? Studios often have computers press the buttons automatically, but checking what happened on screen is still slow.
Sony Interactive Entertainment's filing offers a shortcut. Instead of studying every frame of the gameplay video, the system grabs one thin strip from each frame and stacks the strips into a single still picture. A sudden scene change shows up as a clean horizontal edge. Scrolling shows up as slanted lines.
A program then reads that one picture and compares it with the buttons that were pressed. If the test told the game to walk for three seconds and the picture shows only two seconds of movement, something went wrong.
… create at least one digest image by extracting, from each of a plurality of frame pictures constituting the video, partial pictures of positions different from each other …
Translation: It builds a single summary picture by grabbing different pieces out of multiple video frames.
How Sony stacks pixel rows from every frame
The system takes in two things: a record of the operation input (which button or key was pressed, and when) and the video the screen showed in response. It then builds what the filing calls a digest image, a single still picture made from many video frames.
Here is the example the filing gives. Take a 60 frames-per-second video that is 720 pixels tall. Twelve seconds of it is 720 frames. Grab the top pixel row from frame 1, the second row from frame 2, the third row from frame 3, and so on, then stack them in order. The result is a picture the same size as one frame, and it holds twelve seconds of change. The filing says this resembles a slit-scan image, a photography technique that records a thin slice of a scene over time.
The checking step then looks for lines in that picture, using standard edge-detection tools (software that finds sharp boundaries in an image):
- A line running along the stacking direction marks a scene transition, such as a menu popping up.
- A slanted line marks scrolling, and its angle hints at direction and speed.
- A background that stays flat while only the character changes suggests the character is stuck against an obstacle.
The filing also describes feeding results back so the automatic tester can retry an input, stop pressing, or press the opposite direction to undo an unexpected shift.
… acquires input information describing operation input contents and a video on a screen that is displayed according to the operation input …
Translation: The system collects both what the player did and the resulting gameplay footage.
Why game testers could care about this filing
Testing a game by pressing buttons and watching the result is slow, and the usual fix is writing a custom checker for each game. The filing's pitch is that this method needs few game-specific rules, so one approach could cover many kinds of games. If it works as described, testers could spend less time watching replays and more time fixing what the system flags.
The filing also describes using it with a human player. A controller could vibrate harder, or a lamp near the screen could flash, when your character drifts off the path you input. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. This is a behind-the-scenes tool, so expect no new feature for players.
Sony's 684th filing in our Sony coverage since May adds to a run of 3D work that includes matching characters from flat photos and backlight flashing to reduce ghosting.
Claim 1 is short and wide. It needs a processor, a record of an operation input, a video shown in response, a still image built from pieces taken at different positions in many frames, and a check of the video against the input. It does not say "game." It does not say the input has to be automatic, and it does not mention lines or pixel rows.
The specifics sit further down. Detecting lines is claim 2, scene changes and scrolling are claims 3 to 5, and the loop where the tester picks its next input from the result is claim 6. If claim 1 survived as written, it would cover any tool that stacks slices of video frames and checks them against what a person or program pressed, in games or anywhere else.
That breadth is the interesting part and probably the sticking point. The underlying trick is neat and easy to picture. Whether the claims keep this much room after the examiner's pushback is the real question.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0350870 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in