Sony Patent Links Visual Feature Data Directly to 3D Point Cloud Maps

By Patentlyze Team · Updated Jul 10, 2026

Sony's semiconductor division is patenting a way to combine two things that usually live in separate systems: a 3D map of a physical space and a camera's analysis of what's actually inside it.

What Sony's combined 3D-and-image scanning system does

Imagine a security camera that doesn't just record video but also builds a 3D model of the room it's watching, then labels every person or object in that model with information about what the camera "sees." That's the basic idea here.

Sony's patent describes a system that takes depth-sensing data (the kind of data that produces a point cloud, which is basically a three-dimensional dot-map of a space) and pairs it with analysis from a regular camera image. The analysis might describe a person's posture, an object's appearance, or some other observable feature. Those analysis results get attached directly to the 3D map data.

The combined package is then sent to an external device or system, so whatever receives it gets both the spatial layout and the image-derived context in one go. Instead of handling two separate data streams that have to be matched up later, a receiving system gets them already linked.

How the sensor data and image analysis get linked together

The patent describes four core components working in sequence:

• Point cloud generation: Sensor data (likely from a depth camera or LiDAR) is processed into a point cloud, which is a 3D structure made of millions of individual coordinate points representing the physical layout of a space.
• Feature evaluation: A separate analysis pipeline processes a conventional camera image of the same area and extracts "features" of subjects in the frame. The patent doesn't lock this to one use case, but features could include things like body pose, shape, or other visually derived attributes.
• Association: The feature analysis results are tagged onto the point cloud data, so each evaluated subject in the image has its descriptors linked to its corresponding location in 3D space.
• Output: The merged data package is transmitted to an external apparatus, which receives spatial and contextual information together rather than as separate streams.

The key design choice is the association step: instead of leaving it to downstream systems to reconcile 2D image analysis with 3D spatial data, this patent bakes the linkage into the output format itself. That makes the data immediately usable without additional merging work on the receiving end.

What this means for cameras, robots, and smart sensors

For any system that needs to understand both where things are and what those things are, having that information pre-packaged is a meaningful efficiency gain. Robotics, autonomous vehicles, smart surveillance, and retail analytics all involve exactly that combination, and today those systems often stitch together separate data feeds with extra processing steps.

Sony Semiconductor makes image sensors used in a huge range of devices, from smartphones to industrial cameras. A patent like this suggests the company is thinking about how its sensor hardware can produce richer, more immediately useful data outputs, rather than raw streams that other systems have to interpret entirely on their own. That positions Sony's sensors as capable of doing more of the analytical "work" before the data even leaves the device.

Editorial commentary on a publicly published patent application. Not legal advice.