Sony Patents a Cost-Volume System for High-Accuracy ToF Depth Estimation

By Patentlyze Team · Updated May 29, 2026

Time-of-Flight sensors are fast but fuzzy — they trade spatial resolution for speed. Sony's new patent describes a way to squeeze much more depth accuracy out of that sparse data by treating each pixel's distance as a probability problem, not a single hard number.

What Sony's ToF depth-sharpening technique actually does

Imagine your phone's depth sensor is like a low-res thermal camera — it can tell you roughly how far away things are, but the resulting "depth map" looks blocky and imprecise compared to the sharp color image the main camera captures. That mismatch is a real headache for anything that relies on knowing exactly where objects are in 3D space.

Sony's patent tackles this by building what it calls a cost volume — essentially a data structure that, for every pixel in the image, stores not just one distance guess but a whole probability distribution of plausible distances. Instead of saying "that pixel is 1.2 meters away," it says "here are the odds it's 1.1m, 1.2m, 1.3m..." and works from there.

The result is a depth-estimation process that's better at handling uncertainty, especially at edges where objects at very different depths sit right next to each other. The patent explicitly flags this as a tool for upsampling ToF sensor output — taking coarse depth data and reconstructing a much finer, more accurate depth map.

How cost volumes turn sparse ToF data into dense depth maps

The core of this patent is the cost volume generation unit, a processing module that ingests raw distance measurements from a Time-of-Flight (ToF) sensor — a sensor that fires infrared light pulses and measures how long they take to bounce back — and converts them into a richer representation.

A cost volume is a concept borrowed from stereo-vision research. Rather than producing a single depth value per pixel, it builds a 3D array where one dimension represents candidate depth levels and each cell holds a "cost" (or inversely, a probability) for that depth hypothesis at that pixel location. Think of it as a histogram of depth guesses layered over the entire image.

From this cost volume, a separate distance value estimation device then reads off the most likely depth per pixel — but because it's working from a probability distribution rather than a raw noisy measurement, it can be much more precise, especially near depth discontinuities (object edges).

The patent also references a three-dimensional model generation device downstream, suggesting the pipeline is designed to feed into full scene-reconstruction workflows. The system pairs the ToF sensor with a conventional image sensor, so the high-resolution RGB frame can guide the depth upsampling — a common but technically demanding fusion task.

What this means for 3D sensing in Sony's image pipeline

Sony Semiconductor Solutions supplies image sensors and depth-sensing components to a huge portion of the mobile and automotive markets, so improvements to ToF processing have broad downstream reach. Better depth upsampling directly improves the quality of portrait-mode bokeh, AR object placement, in-cabin driver monitoring, and any robotic or autonomous system that uses a Sony depth module.

The cost-volume framing is also notable because it's the same probabilistic approach that powers high-end stereo-depth networks in research — applying it explicitly to ToF upsampling suggests Sony is pushing its sensor pipeline closer to the accuracy levels you'd normally need dedicated compute for. If this makes it into a real-time embedded implementation, it could raise the bar for what mid-range depth sensors can do.

Editorial commentary on a publicly published patent application. Not legal advice.