New Apple Patent Filters Video Based on Your Gaze Direction
Apple has filed a patent for a system that watches where your eyes point and blurs everything else before sending video over a network, a technique that could cut the bandwidth needed to stream high-quality AR video without you ever noticing the corners are fuzzy.
What Apple's gaze-based video filter actually does
Imagine streaming a live video feed through a pair of AR glasses. Sending every pixel at full quality would eat through your connection quickly, but you actually only need crystal-clear detail in the one spot your eyes are focused on at any given moment. Everything in your peripheral vision can be softer without you noticing.
That's the core idea in this Apple filing. A system tracks where your eyes are pointed, then applies a gentle blur to the parts of the frame you aren't looking at before compressing and sending the video to a processor. The area you're actively looking at gets preserved at full sharpness. The surrounding region gets smoothed out first, which means the compression algorithm has an easier job and sends less data overall.
The end result: the video feed feels full quality to you, because your eye is always in the sharp zone, while the system handles the rest with far fewer bytes.
How the multi-layer filter tracks gaze and compresses selectively
The patent describes a multi-layer low-pass filter (a mathematical tool that removes fine detail, like softening a photo in a photo editor) applied to each video frame before it gets compressed and sent over a network.
The filter has at least two layers:
- Layer one covers the region of the frame directly in line with where the person's gaze is pointing. This zone uses a lighter filtering setting, preserving more detail.
- Layer two covers the area surrounding that central zone. This outer ring gets a heavier blur applied, stripping out fine detail the viewer wouldn't notice anyway because their eyes aren't aimed there.
After filtering, the modified frame is compressed using standard video compression. Because the outer zone is already smoothed, the compression algorithm can reduce the file size more aggressively without introducing obvious visual artifacts. The compressed result is then transmitted over a network to a video processing engine, which reconstructs and displays a modified version of the scene.
The gaze tracking step is key: the filter zones shift in real time as the viewer's eyes move, so the sharp zone always follows the eyes and the blur stays in the periphery.
What this means for AR headsets and video streaming bandwidth
Streaming high-resolution video for AR or mixed-reality devices is a bandwidth problem. Headsets need to send a lot of data quickly to render convincing overlays on a live environment, and that gets expensive on cellular or Wi-Fi connections. A filter that discards detail the viewer can't actually perceive reduces the data load without degrading the experience in any noticeable way.
This technique, often called foveal rendering when applied to graphics, is already used inside some VR headsets to reduce GPU load. Applying it to the transmission side, before the video is even compressed and sent, is a useful extension of the idea. For Apple, which sells the Vision Pro headset and is reportedly working on lighter AR glasses, keeping video streaming efficient over a wireless connection is a real engineering priority.
This is practical, focused engineering work tied directly to a known Apple product category. Foveal compression for streaming isn't a new concept in academic literature, but Apple filing it as a system-level patent with explicit gaze tracking and network transmission steps suggests it's being prepared for a real product pipeline. Worth watching if you follow Vision Pro or any future Apple AR hardware.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.