Samsung Files Patent to Make a Headset's Camera View Match Where You Look
Samsung's latest patent filing tracks where your pupils sit and which way your eyes point, then reshapes a headset's live camera video to match what your eyes would see.
What Samsung's eye-tracking passthrough does
The cameras on a video headset don't sit where your eyes do. They are mounted away from your face and often point somewhere other than where you are looking. So the picture the headset shows you is taken from the wrong spot, and software has to reshape it to look like what your eyes would see.
Samsung's filing adds your eyes to that calculation. Small inward-facing cameras watch where your pupils are and which way each eye points, and the headset reshapes the outside video to fit. Look at something close and your eyes angle inward; look far away and they relax. The picture follows along.
The headset also accounts for where its own cameras are mounted, so the same approach could handle different people and different camera setups.
… determining, using the at least one processing device, positions of pupils of the user's eyes and orientations of the user's eyes based on the second image frames; …
Translation: The headset tracks where your pupils are pointing using internal cameras.
How the headset reads your pupils and gaze
This is a video see-through headset (one that shows you live camera video of the room instead of letting you look through clear lenses). The patent describes two sets of cameras: outward-facing see-through cameras that film the scene, and inward-facing cameras that film your eyes under infrared light from small LEDs.
From the eye footage, the headset works out two things for each eye: where the pupil is and which direction the eye is pointing. It finds these by tracking bright reflections (glints) off the pupil and the cornea, the clear front surface of the eye. It can also measure the distance between your pupils, which shifts as you focus near or far.
Then it builds a transformation for each eye, a grid of numbers (a matrix) that describes how to reshape the image:
- The eye part combines a rotation from your gaze direction with a position from your pupil. It changes as your eyes move.
- The camera part comes from where the cameras sit on the headset, measured during calibration. It stays fixed.
- The two are multiplied together into one matrix per eye, and the description says other standard corrections can be folded in too.
Because everything ends up in a single matrix, each video frame is reshaped once, which the filing says helps efficiency and cuts delay. The description also mentions reusing earlier eye calculations when your eyes barely move, to save computing.
… performing passthrough transformations of the first image frames based on the geometric transformations to generate transformed image frames …
Translation: It mathematically warps the outside video feed to match your natural eye movement.
What eye-aware passthrough means for headsets
If you wear a headset that shows you the room through cameras, the goal is a view that feels like looking through a window. When the picture is built for the wrong eye position, it can drift from what your eyes would naturally see. This patent goes after that mismatch by updating the view as your eyes move, and as a different person puts the headset on.
There is a practical upside for the company too. Folding every correction into one calculation per eye keeps the workload light, which matters on a headset handling video at 30 to 120 frames per second. It is still only a filing, so nothing here says when or whether it shows up in a product.
That makes this Samsung's 78th filing we've tracked in our AR glasses race watchlist since May, adding to work like one on washed-out colors and one on judging distance.
Passthrough on a mixed reality headset works by showing you live camera footage of the real world, adjusted to match where your eyes are actually pointing. When that alignment slips, the world looks subtly wrong, the way a reflection in a window never quite matches what is behind it.
Samsung's filing describes a system that tracks where your eyes are looking and continuously adjusts the footage to stay matched to your gaze, whether you glance at your keyboard, scan across a room, or share the headset with someone whose eyes sit differently than yours.
No test results are included, so the real-world improvement is still an open question. But the problem being solved is the exact thing that makes these headsets feel artificial rather than transparent.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0340769 A1 · click any drawing to enlarge
Show the other 2 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in