Samsung Files Patent to Fix Passthrough Video That Drifts Out of Line With Your Eyes
Headsets that show you the real world through cameras have a built-in problem: the cameras are not where your eyes are. Samsung's filing uses eye tracking to keep correcting for that gap as your eyes move.
What Samsung's eye-tracking passthrough fix does
Every time you glance from a coffee mug on your desk to a window across the room, your eyes shift a little inside a headset. Your pupils move closer together or farther apart, and they sit a bit nearer or farther from the lenses.
In a headset that shows you the outside world through cameras, those tiny shifts matter. The cameras sit on the front of the device, not where your eyes are, so the picture can land slightly off from what your eyes expect. That mismatch can cause eye strain and queasiness.
Samsung's patent application describes a headset that watches your eyes with its own sensors, then re-maps the camera picture to match where your eyes are at that moment. It also fixes color fringing and warping caused by the display lenses.
… applying, using the at least one processing device, a viewpoint matching mapping between viewpoints of image data and viewpoints of a user's eyes based in part on the eye tracking data; …
Translation: The system aligns the camera view with your exact eye position using eye tracking.
How the headset re-maps camera views to your eyes
The filing covers a method for video see-through headsets (also called passthrough headsets), which show a live camera feed of your surroundings, often blended with virtual objects. The device collects three inputs: camera images of the scene, eye tracking data, and head pose data (where your head is and how it is oriented). The description also adds depth data from a sensor such as lidar.
From the eye tracking data, the device works out two measurements. Interpupillary distance (IPD) is the gap between your pupils. Eye relief is the distance from your pupil to the display lens. Both change as your eyes move.
The device then does several things with those numbers:
- Predicts how your head pose will have changed between when a camera frame was captured and when it appears on screen, using the IPD and eye relief changes.
- Builds a viewpoint matching mapping, which translates what the cameras saw into what your eyes would see from where they actually are.
- Corrects parallax error (objects appearing shifted because of the viewing angle) and registration error (virtual objects not lining up with real ones).
- Adjusts a stored lens distortion model to fix warping and chromatic aberration (colored edges), updating it as IPD and eye relief change.
The final frames can be blended with computer-generated objects before being shown on the display.
… correcting parallax error and registration error in the final view frames using the eye tracking data.
Translation: It fixes visual misalignments so the video matches your perspective.
What this means for mixed reality headsets you wear
If you have worn a passthrough headset, you know the world can look slightly wrong: a bit too close, a bit too flat, or swimmy when you move. The filing says mismatches like this cause visual discomfort and motion sickness, and that adapting to your eyes could allow longer comfortable sessions. That is a practical goal for anyone who wants to wear one for work or games.
The approach also matters for mixed reality in particular. A virtual object only feels like it belongs on your real table if it lines up with the table the cameras show, and the registration fix targets exactly that. Samsung lists headsets and smart glasses as example devices, though the patent does not name a product.
That makes this Samsung's 79th filing we've tracked in our AR glasses race watch since May, adding to work like the gaze-matched camera fix and the color washout patch.
The core of this patent is software: it takes sensors a passthrough headset already needs, cameras, head tracking, and eye tracking, and changes how the image is processed before it reaches your eyes. No new hardware is described or required.
What has to exist first is a headset that already tracks your eyes and has a stored profile of how its own lenses bend and color-shift light. The filing assumes both are in place, which means the shortest path to a product is a software update pushed to a device that already ships with those capabilities.
The open question is whether it works fast enough to feel invisible. The document describes predicting head movement to avoid lag, but offers no numbers on how well that prediction holds up in practice. If the engineering lands, the result is a headset where the world simply looks correct, and most people would never think twice about why.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0339431 A1 · click any drawing to enlarge
Show the other 3 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in