Sony Patents a System That Fills In Missing Pieces of 3D Scans
3D scanners routinely leave holes where objects are partially blocked or too far away. Sony has filed a patent for a method that finds those holes and fills them in, while making sure every added point still carries the correct label for what it belongs to.
What Sony's point-cloud gap-filling actually does
Ever tried to piece together a jigsaw puzzle when half the pieces are missing? That is exactly the problem 3D scanners face. When a camera or lidar sensor captures a scene, it records millions of tiny dots in space, called a point cloud. Some objects end up with big gaps because a wall was in the way, the lighting was poor, or the sensor just did not reach far enough.
Sony's patent describes a process that automatically groups those dots into clusters, figures out what real-world object each cluster represents (a chair, a wall, a person), and then adds new dots to fill in the gaps. Crucially, every dot added during the fill-in step gets tagged with the same label as the cluster it belongs to, so nothing gets mislabeled in the process.
The goal is a cleaner, more complete 3D picture of a scene, where every point knows what object it is part of, with no manual cleanup required.
classifying a point cloud into a plurality of clusters indicating lower-layer objects; determining an attribute of an object among the objects for each of the clusters; complementing a cluster among the clusters on a basis of an attribute assigned to the cluster …
Translation: The system groups 3D data points, figures out what each group represents, and fills in any missing parts.
How Sony's system clusters, labels, and fills 3D data
The patent describes four steps that run in sequence on a raw 3D point cloud.
- Clustering: the system groups nearby points together into clusters, each representing a distinct lower-level object in the scene (think of sorting a pile of mixed Lego bricks by color and shape).
- Attribute determination: for each cluster, the system works out what kind of object it represents, assigning a semantic label (chair, floor, vehicle, and so on).
- Cluster complementation: using that label as context, the system identifies where the cluster is incomplete and adds new points to fill the gaps. The label guides what a plausible fill looks like.
- Attribute assignment: every newly added point inherits the same label as the cluster it was added to, so the completed cloud remains consistently annotated.
The key design choice is that the fill step happens after labeling, not before. That ordering means the system can use knowledge about the object type to make smarter guesses about what the missing geometry should look like, and it prevents fill points from being left with no label or the wrong one.
In the attribute assigning processing, the same attribute as that of the cluster is assigned to one or more points added to the cluster by complementation.
Translation: Newly added data points automatically inherit the same labels as the rest of their group.
What this means for 3D scanning and spatial computing
Incomplete 3D scans are a persistent headache for anyone building augmented reality environments, training autonomous vehicles, or doing spatial mapping. A gap in a scan is not just a visual blemish; it can confuse downstream software that relies on complete, labeled geometry to make decisions. Sony's method addresses both problems at once by filling the gap and keeping the label intact.
Sony's steady investment in spatial sensing and 3D capture suggests this kind of foundational data-quality work is supporting a broader push into AR, robotics, and immersive media. For the person actually using a Sony camera or headset to scan a room, the practical payoff would be fewer artifacts and less time fixing scans by hand.
This is the 441st Sony filing in our Sony coverage since May, adding to work like self-steering laser light and defect-cutting laser depth sensing.
When you scan a room with your phone to plan furniture or try on an AR feature, the result often comes back with holes and unlabeled blobs where the software lost track of a surface. This patent describes a method that fills those gaps automatically and stamps every filled patch with the correct label, so a missing section of sofa comes back identified as sofa rather than as anonymous floating geometry.
That label matters because the apps built on top of your scan depend on it. A tool trying to measure your couch or place a virtual lamp on your floor needs to know what it is touching, and a patch of unnamed surface forces it to guess or fail.
The practical result is that a scan becomes usable the moment it finishes, without a cleanup step most people would never bother with. That is the difference between a spatial feature someone uses once and one that becomes a habit.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
13 drawing sheets from US 2026/0268697 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →