Sony Patents a Tile-Based Hierarchical Map System for Real-Time 3D Streaming

By Patentlyze Team · Updated May 22, 2026

Sony is patenting a system that breaks 3D scene data into tiles and resolution layers, streaming only the pieces your current viewpoint actually needs — and updating only the chunks that actually changed.

What Sony's tile-based 3D streaming actually does

Imagine a massive open-world game running on a remote server. Instead of sending you the entire scene every frame, the server sends only the slices of visual data you're actually looking at — like how Google Maps only loads the map tiles near where you're currently zoomed in.

Sony's patent describes a system that stores 3D scene information as layered maps — think stacked grids covering color, surface texture, shininess, and other surface details — each at different levels of detail. Your device requests tiles from whichever layer and zoom level matches your current view. If a piece of the scene changes (say, an explosion re-shapes the terrain), only the affected tiles get updated and re-sent.

The two-way nature matters too: if your device changes something locally, it tells the server, which then updates its own copy of those tiles. This keeps both sides in sync without either one re-transmitting the whole scene.

How the hierarchical reference map update loop works

At its core, this patent describes a hierarchical reference map system — a stack of data grids, each at a different resolution, that together describe what a 3D object or scene looks like. These maps aren't just color data; they also encode surface parameters like normals (which direction a surface faces), roughness, and reflectivity — the raw ingredients a renderer uses to light a scene correctly.

The key innovation is selective, tile-based synchronization. Rather than transmitting an entire scene or re-sending whole maps when anything changes, the system divides each map into tiles and tracks changes at that tile level. When a scene update occurs — or when the system predicts one is coming — only the tiles from the relevant map type, resolution layer, and spatial area are fetched and updated.

This works in both directions:

• The content server pushes updated tiles to the image processing device when the world changes.
• The image processing device notifies the server when local changes occur, triggering a matching server-side update.
• The device selects which resolution layer to request based on current viewpoint — close-up geometry gets high-res tiles, distant areas get lower-res ones.

The result is a bandwidth-efficient, latency-aware pipeline for rendering scenes that live partly on a server and partly on a local device — a classic hybrid cloud rendering architecture.

What this means for cloud gaming and remote rendering

Cloud gaming and remote rendering have always hit a wall with bandwidth: streaming full scene data is wasteful, but sending too little leaves rendering artifacts. Sony's tile-and-layer approach is a practical engineering answer to that tradeoff — it's the same insight that makes map apps feel fast, applied to real-time 3D content.

For PlayStation's cloud gaming ambitions, this kind of system could reduce the raw data pushed to your device on every frame, especially in scenes where most of the world isn't changing. It also opens the door for hybrid rendering setups where your local hardware handles some detail while a server manages the heavy lifting — a pattern that becomes more relevant as Sony expands PlayStation streaming to more devices.

Editorial commentary on a publicly published patent application. Not legal advice.