Apple Patents Tech That Wraps Flat Video Into a 3D Room Around You
Apple is working on a system that takes ordinary flat video, think a movie or a TV show, and automatically reconstructs it as a three-dimensional scene you can stand inside, rather than watch on a screen.

- Publication number
- US 2026/0245315 A1(US20260245315A1)
- Applicant
- Apple Inc.
- Filed · Published
- Apr 13, 2026 · Aug 20, 2026
- Inventors
- Ian M. Richter, Daniel Ulbricht, Jean-Daniel E. Nahmias, Omar Elafifi, Peter Meier
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 345/629
- Parent application
- Continuation of 17/862,301 (filed Jul 11, 2022)(17862301)
What Apple's flat-video-to-3D conversion actually does
You're watching a nature documentary on a spatial headset and the camera pans across a vast canyon, but it looks like a flat rectangle floating in your living room. It pulls you out of the experience completely.
Apple's patent describes a system that could change that. Instead of projecting the video onto a flat surface, the system analyzes the footage, figures out what's in the scene, and rebuilds it as a full 3D environment around you. The canyon walls would stretch to your left and right. The sky would fill above you. You'd be in the scene, not looking at it through a window.
To do this, the system pairs the video analysis with pre-made digital assets, 3D objects, textures, and environments that match what was in the original footage. Those assets get arranged and animated based on what the video actually shows, frame by frame. The result gets displayed through a headset that can already blend digital content with the real world around you.
… performing a scene parsing process on the portion of the pre-existing flat video content to synthesize a scene description for the scene …
Translation: The system analyzes the standard video to figure out what is in it.
How the scene parser rebuilds flat footage in three dimensions
The patent describes a pipeline with a few distinct stages that work together to transform flat, pre-existing video into something spatial.
First, a scene parsing process runs on a segment of the video. Scene parsing means the system analyzes the footage computationally, identifying objects, people, depth relationships, camera movement, and the overall layout of what's on screen. It produces a scene description, essentially a structured data representation of what the scene contains and how it's arranged.
Second, the system pulls in digital assets tied to that video content. These could be 3D character models, environment geometry, or texture maps prepared in advance for a particular film or show. Think of them as the building blocks that can represent the same content in three dimensions.
Third, those assets get driven by the scene description, positioned, animated, and arranged to match what the video's scene parsing determined. The output is a synthesized reality (SR) reconstruction: a fully three-dimensional version of the original scene.
Finally, the reconstruction is presented through a pass-through display device (a headset that shows the user's real physical surroundings blended with digital content). The user experiences the reconstructed scene from within it, spatially, rather than watching a flat image projected in front of them.
… generating a corresponding three-dimensional synthesized reality (SR) reconstruction of the scene by driving the one or more digital assets according to the scene description …
Translation: It builds a 3D environment using computer graphics based on that analysis.
What this means for watching video on Apple Vision Pro
For anyone who owns or might buy a spatial headset like Apple Vision Pro, this patent points at one of the most obvious friction points with the device: ordinary video content looks like a floating rectangle, not an experience. A system that could automatically wrap a movie's environment around you, placing you inside a scene rather than in front of it, would change how the headset handles the majority of available content rather than just the small slice of media produced natively for spatial display.
The practical catch is the dependency on pre-prepared digital assets. The patent describes matching assets to specific video content, which implies studios would need to prepare those assets in advance, making this more of a partnership model than a universal automatic upgrade. That said, the AR category is one of the busier areas in new Big Tech patents right now, and Apple's filing adds specific detail about how scene analysis and asset libraries could slot together to solve the flat-content problem at scale.
Apple's 30th filing we've tracked since May in our AR glasses watch builds on the single-knob lens design and a gaze-sharpening method to keep refining how a headset handles what you see.
Source: US 2026/0245315 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Apple patent news breaks.
We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.
The drawings · 9 sheets
9 drawing sheets from US 2026/0245315 A1 · click any drawing to enlarge
