Sony Patents a Prediction System for Compressing 3D Video Data

By Patentlyze Team · Updated Jul 24, 2026

Streaming a three-dimensional scene, like a sports replay you can walk around in or a lidar map from a self-driving car, takes an enormous amount of data. Sony's new patent describes a smarter prediction system to compress that data without losing quality.

How Sony's compression predicts 3D space occupancy

Imagine trying to send a 3D video of a live concert to someone's headset. A regular video is just flat pixels, but a 3D scene has millions of tiny floating points in space, each needing coordinates. That adds up fast, so you have to compress it aggressively, and doing that badly turns the scene into a blurry mess.

Sony's patent addresses this by teaching a system to predict where points in 3D space are likely to be, before it finishes decoding them. If the system already knows a certain region of space was occupied in the previous video frame, it can make a confident guess that the same region is probably occupied in the current frame too. Better guesses mean you need fewer bits to describe what's actually there.

The trick is combining several of these guesses, from neighboring spots in the current frame, the same spot in earlier frames, and nearby spots in earlier frames, and then weighing each one by how useful it's likely to be. That weighted combination drives the final compression step.

How the composite vector weights spatial and time neighbors

The patent covers a compression pipeline for 3D point cloud data, which is how lidar scanners, volumetric cameras, and many spatial computing systems represent physical space as clouds of coordinate points rather than flat pixels.

The data is organized as an octree (a tree structure that divides 3D space into progressively smaller boxes, or nodes). The system's job at each node is to predict which child boxes, the smaller subdivisions inside it, actually contain any points. This yes/no answer for each child box is called its occupancy state.

• Context vectors: short numerical summaries of what neighboring regions look like, pulled from the current frame, the same layer of a neighboring frame, and a lower (finer detail) layer of a neighboring frame.
• Importance coefficient vector: a set of weights that controls how much each context vector contributes to the final prediction. This is the key idea: not all neighbors are equally informative, so this weighting step stops unhelpful context from muddying the prediction.
• Composite vector and predicted probability vector: the weighted combination is fed through a model to produce probability scores for each possible occupancy pattern. Those scores drive an arithmetic coder (a standard compression tool) to encode the actual data very efficiently.

The system can draw on within-frame spatial patterns, across-frame temporal patterns, or both, depending on what's available and useful.

What this means for streaming 3D and volumetric video

Point cloud compression is one of the core engineering problems standing between today's clunky volumetric video demos and a world where you can actually stream a 3D sports event or a real-time lidar feed over a normal internet connection. Every efficiency gain directly translates into lower bandwidth costs or higher visual fidelity at the same data rate.

Sony is a major player in both professional video equipment and consumer electronics, and the company has been active in spatial and immersive video standards bodies. A patent like this positions Sony to contribute to, or potentially collect royalties from, next-generation point cloud coding standards like V-PCC and G-PCC, the formats being developed for exactly this kind of content.

Editorial commentary on a publicly published patent application. Not legal advice.