Samsung Files Patent to Fill Gaps in How Headsets Judge Distance
Samsung's patent application tackles an odd gap in mixed reality headsets: their depth sensors measure distance at only a scattering of points, so the headset has to guess the rest. The proposed fix uses the regular color photo to keep object edges sharp while it fills in the blanks.
What Samsung's headset depth-filling patent does
Headsets that show you the real world through their cameras have a distance problem. Their depth sensors return only a sparse scattering of readings, with big blank patches in between, yet the headset needs a distance for every single pixel to redraw the scene from your eyes' point of view. Get it wrong and, as Samsung's filing puts it, you see inaccurate and disconcerting frames.
Samsung's patent application describes a cleanup routine. The headset blends distance readings from its depth sensor with ones worked out from pairs of camera images, makes a rough guess for the empty pixels, and tracks how much it trusts each number. Then it studies the ordinary color picture to find where objects begin and end, and uses those outlines to fix the shaky guesses without blurring edges.
The end product is a full distance map as detailed as the camera image itself, ready for the headset to build the final view you see.
… performing depth densification and depth hole filling, using the at least one processing device, the depth edge preserving filter, and the parameters, to create a dense depth map of the image frame; …
Translation: The headset fills in missing distance data to create a complete map of the room.
How color images guide the missing distance fixes
The claims describe a pipeline that runs on the headset's own processor, from raw camera and sensor data to the finished frame:
- Collect: color frames from the see-through cameras, plus depth readings from a depth sensor (the filing lists time-of-flight, lidar and structured light sensors).
- Fuse and fill: sensor depth is combined with depth worked out from stereo image pairs (two cameras seeing the scene from slightly different spots) or from structure from motion (estimating depth as the camera moves). Empty pixels get a rough guess by interpolation (averaging nearby values). Every pixel gets a confidence score: sensor readings get the top score of 1, and pixels with no depth get 0.
- Build the filter: in small overlapping windows, the system models depth as a simple linear function of the image's color (an affine transform, meaning a scale and an offset), so depth changes follow color edges.
- Refine: depths at low-confidence pixels are recomputed and noise is removed. Overlapping windows give several estimates per pixel, which are averaged, then checked.
- Render: the dense map drives the passthrough transformation, which shifts the camera's view to match your eyes and compensates for head movement, before lens correction and blending in virtual objects.
The filing also says a machine-learning model could produce the first rough depth map, with the edge-preserving filter then polishing it.
Why clean depth matters for Samsung's headsets
If a headset misjudges distance, the real world on your display can look slightly off: objects sit in the wrong place, edges wobble, and virtual items can appear in front of things they should be behind. The filing links accurate depth to better visual comfort and less motion sickness, and says filled-in depth lets real objects properly block virtual ones.
The design runs on the headset itself, with no outside computing, and reuses the color cameras already on the device instead of adding sensors. For anyone wearing a mixed reality headset for long stretches, that kind of low-latency cleanup could decide whether the passthrough view feels natural or tiring. The filing names headsets and smart glasses as target devices.
That makes this Samsung's 76th filing we've tracked since May in our AR glasses race watchlist, building on earlier applications like one hiding AR with a dial and selective 3D by gaze.
Passthrough headsets succeed or fail on whether the room looks right, and that depends on knowing how far away every pixel is. Depth sensors give only sparse readings, so the gap between what the headset measures and what it needs is large and unavoidable. The problem is real, and it shows up as wobbly edges and queasy wearers.
The approach fits the size of the problem. Filling gaps by borrowing edge information from the high-resolution color image uses data the headset already has, and the confidence scores tell it which numbers to trust and which to rework. That is a sensible way to spend limited battery and processing power.
The filing gives no test results, frame rates or error figures, so how much better the picture gets is unknown. It reads as a careful piece of engineering aimed at a problem every headset maker faces, and it makes no leap beyond that.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
13 drawing sheets from US 2026/0340959 A1 · click any drawing to enlarge
Show the other 5 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in