Waymo Patents Technology That Shows Any Street Scene From a New Camera Angle
Waymo is working on a way to generate photorealistic images of roads and environments from camera angles that never actually existed, which could let its autonomous vehicles train on far more driving scenarios than any camera crew could ever physically capture.
What Waymo's scene-rendering system actually does
You're driving down a city street, and a camera on the roof captures the scene from one fixed angle. But what if the car's AI needed to know what that same moment looked like from six inches to the left, or from a meter higher up? Physically repositioning a camera after the fact is impossible.
That's the problem Waymo is working on with this patent. It describes a system that takes footage from existing cameras and uses a neural network (a type of AI trained on patterns) to mathematically reconstruct the full three-dimensional scene. Once that reconstruction exists, the system can generate a synthetic but realistic image from any new camera position you specify, even one that was never physically there.
For a self-driving car company, that's a big deal. Instead of having to drive every road in every possible configuration, Waymo could use real footage to generate thousands of training images showing slightly different perspectives, lighting conditions, or object positions, giving its AI far more to learn from.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rendering a new image that depicts a scene from a perspective of a camera at a new camera viewpoint.
Translation: Software generates a fresh picture showing a street from the exact angle of a newly positioned camera.
How neural view synthesis rebuilds a scene's geometry
The patent covers a method for neural view synthesis, the process of using a neural network to render a new image of a scene from a camera position that was never physically occupied.
At a high level, the system works like this:
- It takes one or more existing images captured by real cameras, along with data about where those cameras were positioned and which direction they were pointing.
- A neural network processes that input and builds an implicit 3D representation of the scene, essentially a mathematical model of what the space looks like in all directions, including surfaces, colors, and how light bounces around.
- Given a new, hypothetical camera position (called the new viewpoint), the system uses that 3D model to render what a camera there would actually see.
The patent specifically emphasizes handling large scenes, which is a meaningful technical distinction. Most neural rendering research focuses on small, bounded objects (a chair, a face). Outdoor driving environments are far more complex: they stretch for hundreds of meters, contain dozens of moving and static objects, and vary dramatically in lighting. The system described here is designed to handle that scale.
The filing lists several well-known researchers in neural rendering as inventors, including contributors to NeRF (Neural Radiance Fields), a technique that became a landmark in AI-generated 3D imagery when it was published in 2020.
What this means for how Waymo trains its self-driving AI
For Waymo, the practical payoff is in training data. Self-driving systems need to see millions of driving scenarios to learn how to handle edge cases: a pedestrian stepping off a curb at an unusual angle, glare from a low sun, a truck blocking part of a lane. Physically collecting all of that footage is enormously expensive and slow. A system that can synthesize new camera views from existing footage could multiply the usefulness of every real-world drive the fleet records.
This filing also signals where Waymo is putting serious research effort: not just in perception hardware or mapping, but in the underlying AI infrastructure that makes simulated training more realistic. Autonomous vehicle development increasingly depends on how well a company can close the gap between simulated and real-world data, and filings like this one sit alongside other new tech patents in neural rendering and simulation that suggest this gap is becoming a core competitive frontier.
Google's 47th filing we've tracked since May in our self-driving sensor race watchlist connects to earlier work on blind spot sharing and adjustable sensor fields, adding to how cars might perceive the world around them.
Claim 1 is listed as canceled, which is the most important detail in this filing and the one most worth sitting with. A canceled independent claim means the scope of what Waymo is actually claiming here is unknown from this publication alone. The dependent claims (not provided) may carve out something narrow, or a continuation may refile with broader language. Arguing from what claim 1 actually covers is impossible when claim 1 doesn't exist in this version, so what this filing mostly tells us is that Waymo's legal team is actively shaping and reshaping the claim boundaries around neural view synthesis for large outdoor scenes, which is itself a signal of how contested this technical territory is.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0245351 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →