Sony Patents a System That Reads On-Screen Text So You Don't Have To
Sony has patented a way for a PlayStation system to automatically find and read text that appears on screen during gameplay or video, without a human telling it where to look or when to start scanning.
What Sony's automatic in-game text reader actually does
Ever tried to find the subtitle or on-screen message you missed while gaming? Reading text that flashes on a screen is something humans do easily, but for a machine it takes real effort, especially if it has to check every single frame of video.
Sony's patent describes a smarter approach. Instead of scanning every frame, the system samples the video periodically, looking only in areas where text typically shows up. It then filters the image down to colors that text is usually printed in, checks whether anything text-shaped is big enough to be worth reading, and only fires off a full OCR (optical character recognition, the same technology that lets your phone scan a receipt) scan once it sees that text has been sitting there steadily across several frames in a row.
The result: a system that can pull words off your screen automatically, without burning through processing power on every frame or on every flicker that looks vaguely like a letter.
… a region modification unit configured to apply one or more color ranges to the identified regions such that colors outside of the color ranges are set to a default value, wherein the color ranges are defined based upon expected text colors; …
Translation: It filters out background colors so it can focus just on the text.
How the system decides when to run a full text scan
The patent describes a pipeline with several distinct stages, each designed to avoid doing expensive work until it is actually needed.
- Frame sampling: Instead of looking at every frame of video, the system grabs every nth frame (for example, every 5th or 10th), keeping the workload low during normal gameplay.
- Region identification: Within each sampled frame, the system identifies areas where text is likely to appear, such as subtitle bars, HUD corners, or dialog boxes, rather than scanning the whole image.
- Color filtering: The identified regions are stripped of colors that fall outside a defined range of expected text colors. Pixels that don't match go to a default (blank) value, cutting through background noise.
- Bounding box check: The system looks for clusters of text-like shapes and measures them. If no cluster meets a minimum size threshold, the system moves on to the next sampled frame without doing anything further.
- Persistence check: If a large enough text region does appear, the system switches from sampled frames to consecutive frames and counts how many frames in a row that region holds. Only after a set number of consecutive hits does it trigger the full OCR read.
- OCR output: The recognized text is then output, presumably for accessibility features, parental controls, content moderation, or search indexing.
The persistence check is the key design choice. It prevents stray visual noise or a single-frame logo from triggering a full scan, saving compute while still catching text that genuinely stays on screen.
A system and method for obtaining a plurality of image frames from video content during presentation, the plurality of image frames comprising every n th frame of the content where n is an integer greater than one, …
Translation: It skips certain video frames to save processing power while scanning for words.
What this means for accessibility and game text detection
For players who rely on accessibility features, a system like this could feed on-screen text to a screen reader or translation tool automatically, without the game developer having to build that support in. That matters because many older games and some newer ones never expose their text as readable data, only as pixels burned into the image.
From a platform perspective, Sony could use this to power content moderation (catching on-screen text in user-generated video), parental controls, or search inside recorded gameplay clips. The claim is written broadly enough to cover any video content presented on the system, not just games, which means it could apply to streaming apps or user-uploaded video running on a PlayStation console.
Sony's 483rd filing in our Sony coverage since May adds to a growing audio thread that includes sudden-sound canceling and spatial audio compression.
Claim 1 requires four things to happen together: sample every nth frame from a video, find areas where text might appear, confirm those areas stay above a minimum size across several frames in a row, and only then run the reading process. All four steps must be present for something to fall inside the claim.
The consecutive-frame requirement is where the claim does its real work. Before any reading happens, a text region has to hold its size across multiple frames in sequence, which filters out fleeting on-screen words like a score flash or a passing sign and focuses processing only on stable, sustained text.
That specificity defines exactly what the claim would block if granted: any automatic text-reading feature in a video or gaming platform that follows this precise four-step sequence. A design that skips the multi-frame size check, or runs the reading process before confirming persistence, falls outside it entirely.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0279086 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →