Google Files Patent for AI That Builds 3D Scenes From Flat Photos
Google has filed a patent application for teaching an AI to rebuild a single flat photo as a 3D scene you can look around in. The twist is that it learns from everyday images, with no special 3D training data.

- Publication number
- US 2026/0368998 A1(US20260368998A1)
- Applicant
- Google LLC
- Filed · Published
- May 15, 2025 · Nov 19, 2026
- Inventors
- Deqing Sun, Pratul Srinivasan, Kyle Sargent, Jing Yu Koh, Huiwen Chang, Han Zhang, Charles Herrmann
- Status when we published
- Examiner raised objections; the company can respond
- Document
- 22 claims
- US classification
- 345/419
- Examiner
- WANG, YI (Art Unit 2619)
- Parent application
- National Stage Entry of PCT/US2023/081079 (filed Nov 27, 2023)(PCTUS2023081079)
What Google's photo-to-3D training does
A photo of a person popping a wheelie on a motorcycle sits flat on a screen. You can see it from one angle only, and that's the end of it. Google's patent application describes a way to teach an AI to rebuild a scene like that in 3D, so a virtual camera can move around it.
The hard part is the teaching. Earlier approaches needed special training photos that came with exact camera positions and depth measurements, and those are expensive to collect. Google's idea is to let a separate, ready-made AI estimate the depth in ordinary photos, then use those estimates as the answer key.
That means the model can learn from huge, messy collections of everyday pictures, like the million-plus images in the ImageNet library. The filing says it can also invent brand new 3D scenes from scratch, not just rebuild ones it was shown.
… comparing, using the one or more processors, the first plurality of depth values to the second plurality of depth values to generate a depth loss value for the given first training example …
Translation: The system checks its depth calculations against known values to measure how wrong it was.
How the two-stage training teaches depth
The filing describes training a model in two stages. In stage one, an encoder squeezes a photo into a compact code, and a decoder rebuilds the photo from that code. The decoder is a NeRF (Neural Radiance Field), a type of AI that works out how light would look from any point in a 3D space, so it can show a scene from new angles.
Claim 1 centers on how the decoder gets graded. It has to pass four checks at once:
- Rebuild the original photo accurately (a reconstruction score).
- Fool a judge that tries to tell real photos from rebuilt ones.
- Fool a second judge that compares the rebuilt image against a view of the same scene from a different angle.
- Guess distances that match the depth estimates from an off-the-shelf depth model.
In stage two, the decoder is frozen. A second AI, an autoregressive transformer (a model that predicts one piece at a time, like autocomplete), learns the sequences of codes from stage one. Feed it random noise and it produces a new code, which the decoder turns into a fresh 3D scene. The example uses triplanes (three flat grids of features that together describe 3D space) sized 512 by 512.
… rather than requiring exact ground truth data such as ground truth depth values or tuning pose hyperparameters …
Translation: It works using estimated depth instead of needing manually measured 3D data.
What this means for games, film and photos
The filing's background says making 3D content by hand takes a lot of time, and that existing AI shortcuts depend on rare, costly training data. Learning from ordinary photos could widen what kinds of scenes and objects such a model can handle. The filing also says one pass through the trained model turns a single photo into a 3D scene you can move a camera around, with no slow per-image fitting step.
The tests in the filing report up to a threefold improvement on a standard image-quality score (called FID) over other models on ImageNet. Those are the company's own results. The claims were rejected in a first review, which is routine at this stage and usually followed by amendments, so the final wording may change.
This is the 22nd Google patent we've tracked in our AI simulation filings since May, a run that includes one on a personal AI video channel and one on turning text into 3D objects.
Source: US 2026/0368998 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 15 sheets
15 drawing sheets from US 2026/0368998 A1 · click any drawing to enlarge
Show the other 7 sheets

No comments yet