Sony Patents a Two-Pass System for Reading LiDAR Sensor Data Faster
Sony has filed a patent for a LiDAR image-analysis system that only does the expensive, high-accuracy work on parts of a scene it isn't confident about, skipping the hard pass everywhere else.
How Sony's LiDAR scanner avoids doing double the work
A self-driving car's laser sensor sweeps the road ahead dozens of times per second, turning everything it sees into a point cloud image. Sorting that image into labeled objects (road, pedestrian, bicycle, building) is called segmentation, and doing it both fast and accurately at the same time is genuinely difficult.
Sony's patent describes a two-step approach. A quick, lightweight scan runs first and labels as much of the scene as it can. Crucially, it also reports how confident it is about each area. Anywhere confidence falls below a set threshold, a second, more thorough pass takes over and re-examines just that region.
The result: your device doesn't burn computing power re-checking every pixel when it already knows what most of them are. Only the uncertain patches get the expensive treatment. That keeps processing quick, memory use low, and accuracy high where it actually counts.
… a second processing unit configured to perform second segmentation having higher accuracy than accuracy of the first segmentation on an image region in which a reliability related to a result of the first segmentation is lower than a threshold value …
Translation: A second processor re-analyzes only the messy or unclear parts of the image to ensure high quality.
How the two-stage segmentation pipeline decides what to recheck
The patent describes an information processing apparatus built around three units working in sequence.
- First processing unit: runs a fast, lower-accuracy segmentation pass on the full LiDAR image. It labels each region (road surface, object, background, etc.) and outputs a reliability score alongside each label. That score is essentially a confidence rating: how sure is the model about this particular patch?
- Second processing unit: receives only the regions where the reliability score falls below a configurable threshold. It then performs a more computationally intensive, higher-accuracy segmentation pass on those patches. The second pass doesn't touch anything the first pass was already confident about.
- Estimation unit: stitches the two sets of results together into a single output, combining the quick labels from the first pass with the refined labels from the second pass.
The input is specifically a LiDAR image (Light Detection and Ranging, a laser-based sensor that measures distance by timing how long a pulse takes to bounce back). LiDAR produces dense, structured range data rather than color images, which means ambiguous regions tend to cluster at object edges and at longer ranges where the sensor's point cloud thins out. The threshold-gating approach is designed to concentrate effort precisely there.
… to improve a processing speed while ensuring accuracy of semantic segmentation, and further reduce a memory consumption amount …
Translation: The system speeds up data processing and saves memory while keeping image recognition accurate.
What this means for self-driving car processing costs
For anyone riding in or depending on an autonomous vehicle, perception speed and accuracy are the core reliability trade-off. A system that runs a single fast scan can miss objects at the edges of its confidence; a system that runs a full high-accuracy scan on every frame burns through computing resources and introduces delay. Sony's design aims to thread that needle by making the expensive work selective.
Sony's track record in automotive-sensing patents suggests this fits a broader hardware ambition beyond consumer electronics. Lower memory consumption and faster cycle times matter most in embedded systems with strict power and heat limits, exactly the kind of chip constraints found in production vehicles. If this approach works as described, it means more processing headroom for other safety-critical tasks running on the same hardware.
Sony's 25th filing we've tracked since July in the self-driving sensing race builds on its two-direction depth scanner and a three-sensor alignment method.
This patent solves a real resource problem in a straightforward way. The two-pass confidence-gating idea has precedent in other computer-vision fields, so Sony isn't inventing the concept from scratch. What the patent does is apply it to LiDAR segmentation specifically and tie it to memory savings, which is a practical constraint in real hardware.
The honest payoff for a passenger or pedestrian is indirect: fewer compute bottlenecks on the sensor chip could mean fewer moments where the car's perception pipeline falls behind real-world events. That's a failure you'd never notice if it works, but you'd notice hard if it didn't.
This reads as solid engineering work rather than a conceptual leap. The interesting question is how well the threshold tuning holds up across different environments (rain, fog, dense urban intersections) where LiDAR confidence patterns change unpredictably.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
28 drawing sheets from US 2026/0289790 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in