Get the newsletter
Apple · AR / VR · Filed Apr 13, 2026 · USPTO published Aug 20, 2026 · Aug 20 release

Apple Patents Tech That Wraps Flat Video Into a 3D Room Around You

Apple is working on a system that takes ordinary flat video, think a movie or a TV show, and automatically reconstructs it as a three-dimensional scene you can stand inside, rather than watch on a screen.

Perspective view of converting a flat video frame into a three-dimensional room environment. Drawing from patent filing US 2026/0245315 A1.
Perspective view of converting a flat video frame into a three-dimensional room environment.All 9 drawings ↓
Publication number
US 2026/0245315 A1(US20260245315A1)
Applicant
Apple Inc.
Filed · Published
Apr 13, 2026 · Aug 20, 2026
Inventors
Ian M. Richter, Daniel Ulbricht, Jean-Daniel E. Nahmias, Omar Elafifi, Peter Meier
Status when we published
Waiting for an examiner
Document
20 claims
US classification
345/629
Parent application
Continuation of 17/862,301 (filed Jul 11, 2022)(17862301)

What Apple's flat-video-to-3D conversion actually does

You're watching a nature documentary on a spatial headset and the camera pans across a vast canyon, but it looks like a flat rectangle floating in your living room. It pulls you out of the experience completely.

Apple's patent describes a system that could change that. Instead of projecting the video onto a flat surface, the system analyzes the footage, figures out what's in the scene, and rebuilds it as a full 3D environment around you. The canyon walls would stretch to your left and right. The sky would fill above you. You'd be in the scene, not looking at it through a window.

To do this, the system pairs the video analysis with pre-made digital assets, 3D objects, textures, and environments that match what was in the original footage. Those assets get arranged and animated based on what the video actually shows, frame by frame. The result gets displayed through a headset that can already blend digital content with the real world around you.

From the filing · CLAIM 1
… performing a scene parsing process on the portion of the pre-existing flat video content to synthesize a scene description for the scene …

Translation: The system analyzes the standard video to figure out what is in it.

How the scene parser rebuilds flat footage in three dimensions

The patent describes a pipeline with a few distinct stages that work together to transform flat, pre-existing video into something spatial.

First, a scene parsing process runs on a segment of the video. Scene parsing means the system analyzes the footage computationally, identifying objects, people, depth relationships, camera movement, and the overall layout of what's on screen. It produces a scene description, essentially a structured data representation of what the scene contains and how it's arranged.

Second, the system pulls in digital assets tied to that video content. These could be 3D character models, environment geometry, or texture maps prepared in advance for a particular film or show. Think of them as the building blocks that can represent the same content in three dimensions.

Third, those assets get driven by the scene description, positioned, animated, and arranged to match what the video's scene parsing determined. The output is a synthesized reality (SR) reconstruction: a fully three-dimensional version of the original scene.

Finally, the reconstruction is presented through a pass-through display device (a headset that shows the user's real physical surroundings blended with digital content). The user experiences the reconstructed scene from within it, spatially, rather than watching a flat image projected in front of them.

From the filing · THE ABSTRACT
… generating a corresponding three-dimensional synthesized reality (SR) reconstruction of the scene by driving the one or more digital assets according to the scene description …

Translation: It builds a 3D environment using computer graphics based on that analysis.

What this means for watching video on Apple Vision Pro

For anyone who owns or might buy a spatial headset like Apple Vision Pro, this patent points at one of the most obvious friction points with the device: ordinary video content looks like a floating rectangle, not an experience. A system that could automatically wrap a movie's environment around you, placing you inside a scene rather than in front of it, would change how the headset handles the majority of available content rather than just the small slice of media produced natively for spatial display.

The practical catch is the dependency on pre-prepared digital assets. The patent describes matching assets to specific video content, which implies studios would need to prepare those assets in advance, making this more of a partnership model than a universal automatic upgrade. That said, the AR category is one of the busier areas in new Big Tech patents right now, and Apple's filing adds specific detail about how scene analysis and asset libraries could slot together to solve the flat-content problem at scale.

Apple's 30th filing we've tracked since May in our AR glasses watch builds on the single-knob lens design and a gaze-sharpening method to keep refining how a headset handles what you see.

Source: US 2026/0245315 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Apple patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 9 sheets

9 drawing sheets from US 2026/0245315 A1 · click any drawing to enlarge

Converting flat video to 3D, drawing sheet 1 from US 2026/0245315 A1
Sheet 1 Converting flat video to 3D
User in spatial environment setup, drawing sheet 2 from US 2026/0245315 A1
Sheet 2 User in spatial environment setup
Content generation server architecture, drawing sheet 3 from US 2026/0245315 A1
Sheet 3 Content generation server architecture
Headset device internal architecture, drawing sheet 4 from US 2026/0245315 A1
Sheet 4 Headset device internal architecture
Data processing and reconstruction pipeline, drawing sheet 5 from US 2026/0245315 A1
Sheet 5 Data processing and reconstruction pipeline
Scene generation process workflow, drawing sheet 6 from US 2026/0245315 A1
Sheet 6 Scene generation process workflow
Plot-effector extraction process flow, drawing sheet 7 from US 2026/0245315 A1
Sheet 7 Plot-effector extraction process flow
Spatial content generation workflow, drawing sheet 8 from US 2026/0245315 A1
Sheet 8 Spatial content generation workflow
Show sheet 9
Apple Patents Tech That Wraps Flat Video Into a 3D Room Around You patent drawing, sheet 9
Sheet 9
Patent filing page

Watchlists this filing belongs to

All watchlists →