Get the newsletter
Google · AI / ML · Filed May 15, 2025 · USPTO published Nov 19, 2026 · This week's release

Google Files Patent for AI That Builds 3D Scenes From Flat Photos

Google has filed a patent application for teaching an AI to rebuild a single flat photo as a 3D scene you can look around in. The twist is that it learns from everyday images, with no special 3D training data.

An AI system turns a single flat photo into a full 3D object that can be viewed and rotated from new angles. Drawing from patent filing US 2026/0368998 A1.
An AI system turns a single flat photo into a full 3D object that can be viewed and rotated from new angles.All 15 drawings ↓
Publication number
US 2026/0368998 A1(US20260368998A1)
Applicant
Google LLC
Filed · Published
May 15, 2025 · Nov 19, 2026
Inventors
Deqing Sun, Pratul Srinivasan, Kyle Sargent, Jing Yu Koh, Huiwen Chang, Han Zhang, Charles Herrmann
Status when we published
Examiner raised objections; the company can respond
Document
22 claims
US classification
345/419
Examiner
WANG, YI (Art Unit 2619)
Parent application
National Stage Entry of PCT/US2023/081079 (filed Nov 27, 2023)(PCTUS2023081079)

What Google's photo-to-3D training does

A photo of a person popping a wheelie on a motorcycle sits flat on a screen. You can see it from one angle only, and that's the end of it. Google's patent application describes a way to teach an AI to rebuild a scene like that in 3D, so a virtual camera can move around it.

The hard part is the teaching. Earlier approaches needed special training photos that came with exact camera positions and depth measurements, and those are expensive to collect. Google's idea is to let a separate, ready-made AI estimate the depth in ordinary photos, then use those estimates as the answer key.

That means the model can learn from huge, messy collections of everyday pictures, like the million-plus images in the ImageNet library. The filing says it can also invent brand new 3D scenes from scratch, not just rebuild ones it was shown.

From the filing · CLAIM 1
… comparing, using the one or more processors, the first plurality of depth values to the second plurality of depth values to generate a depth loss value for the given first training example …

Translation: The system checks its depth calculations against known values to measure how wrong it was.

How the two-stage training teaches depth

The filing describes training a model in two stages. In stage one, an encoder squeezes a photo into a compact code, and a decoder rebuilds the photo from that code. The decoder is a NeRF (Neural Radiance Field), a type of AI that works out how light would look from any point in a 3D space, so it can show a scene from new angles.

Claim 1 centers on how the decoder gets graded. It has to pass four checks at once:

  • Rebuild the original photo accurately (a reconstruction score).
  • Fool a judge that tries to tell real photos from rebuilt ones.
  • Fool a second judge that compares the rebuilt image against a view of the same scene from a different angle.
  • Guess distances that match the depth estimates from an off-the-shelf depth model.

In stage two, the decoder is frozen. A second AI, an autoregressive transformer (a model that predicts one piece at a time, like autocomplete), learns the sequences of codes from stage one. Feed it random noise and it produces a new code, which the decoder turns into a fresh 3D scene. The example uses triplanes (three flat grids of features that together describe 3D space) sized 512 by 512.

From the filing · THE ABSTRACT
… rather than requiring exact ground truth data such as ground truth depth values or tuning pose hyperparameters …

Translation: It works using estimated depth instead of needing manually measured 3D data.

What this means for games, film and photos

The filing's background says making 3D content by hand takes a lot of time, and that existing AI shortcuts depend on rare, costly training data. Learning from ordinary photos could widen what kinds of scenes and objects such a model can handle. The filing also says one pass through the trained model turns a single photo into a 3D scene you can move a camera around, with no slow per-image fitting step.

The tests in the filing report up to a threefold improvement on a standard image-quality score (called FID) over other models on ImageNet. Those are the company's own results. The claims were rejected in a first review, which is routine at this stage and usually followed by amendments, so the final wording may change.

This is the 22nd Google patent we've tracked in our AI simulation filings since May, a run that includes one on a personal AI video channel and one on turning text into 3D objects.

Source: US 2026/0368998 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 15 sheets

15 drawing sheets from US 2026/0368998 A1 · click any drawing to enlarge

Hardware parts that run the 3D generation system, drawing sheet 1 from US 2026/0368998 A1
Sheet 1 Hardware parts that run the 3D generation system
Connected computers sharing the work across a network, drawing sheet 2 from US 2026/0368998 A1
Sheet 2 Connected computers sharing the work across a network
Turning a single snapshot into a full 3D object, drawing sheet 3 from US 2026/0368998 A1
Sheet 3 Turning a single snapshot into a full 3D object
Training checks that ensure realistic depth and novel angles, drawing sheet 4 from US 2026/0368998 A1
Sheet 4 Training checks that ensure realistic depth and novel angles
Quality test scores comparing rival image generators, drawing sheet 5 from US 2026/0368998 A1
Sheet 5 Quality test scores comparing rival image generators
Sample pictures and depth maps generated by different tools, drawing sheet 6 from US 2026/0368998 A1
Sheet 6 Sample pictures and depth maps generated by different tools
Rebuilt 3D shapes extracted from everyday photos, drawing sheet 7 from US 2026/0368998 A1
Sheet 7 Rebuilt 3D shapes extracted from everyday photos
More photos converted into 3D models with depth maps, drawing sheet 8 from US 2026/0368998 A1
Sheet 8 More photos converted into 3D models with depth maps
Show the other 7 sheets
Shifting camera angles reveals hidden details in a 3D scene, drawing sheet 9 from US 2026/0368998 A1
Sheet 9 Shifting camera angles reveals hidden details in a 3D scene
Adding depth calculations sharpens 3D shapes across several models, drawing sheet 10 from US 2026/0368998 A1
Sheet 10 Adding depth calculations sharpens 3D shapes across several models
Removing key training steps harms the quality of 3D shapes, drawing sheet 11 from US 2026/0368998 A1
Sheet 11 Removing key training steps harms the quality of 3D shapes
Tuning sampling settings produces clearer generated pictures, drawing sheet 12 from US 2026/0368998 A1
Sheet 12 Tuning sampling settings produces clearer generated pictures
Test results show competitive image quality on car photos, drawing sheet 13 from US 2026/0368998 A1
Sheet 13 Test results show competitive image quality on car photos
How training teaches the AI to estimate depth and realism, drawing sheet 14 from US 2026/0368998 A1
Sheet 14 How training teaches the AI to estimate depth and realism
Two training steps teach the AI complete 3D scenes, drawing sheet 15 from US 2026/0368998 A1
Sheet 15 Two training steps teach the AI complete 3D scenes
Patent filing page

Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.