Google Files Patent to Make Lifelike 3D Scenes Run Smoothly on Your Phone
Lifelike 3D scenes built from ordinary photos usually need a powerful computer to view. Google's filing shows a way to draw them with the same basic graphics tricks video games have used for years.
What Google's phone-friendly 3D scene viewer does
Ever tried to spin around a 3D model of a room on your phone, only to watch it stutter? That is the problem with a newer kind of 3D scan called a neural radiance field, or NeRF. AI turns a set of photos into a scene you can look around in, but showing it means running a big AI model hundreds of times for every pixel on screen.
Google's filing takes a shortcut. It stores the scene as a shell of flat shapes, like a paper-mache sculpture of the room, with extra information painted onto each surface. Your phone's graphics chip draws that shell the way it draws a video game, and a tiny AI then turns the painted-on information into the final color of each pixel, based on the angle you're looking from.
The filing says this runs about 10 times faster than leading methods at the same picture quality, and it can run in an ordinary web browser.
… a machine-learned neural fragment shader configured to process the specified camera pose and the respective learned feature and opacity data for each fragment of the feature image to output a respective output color for each fragment for use in the rendered image.
Translation: An artificial intelligence component colors each piece of the 3D scene based on the current camera angle.
How flat shapes and a tiny AI paint each pixel
The claims cover a rendering pipeline (the sequence of steps that turns stored 3D data into an image on screen). It starts with a mesh model, a collection of flat polygons that roughly follow the surfaces in a scene. It also uses texture images that hold learned data for each polygon: a yes-or-no opacity value and a short list of numbers called a feature vector.
To draw a frame from a given camera position, the pipeline runs two steps:
- A mesh rasterizer (the standard graphics step that works out which surfaces are visible) produces a feature image, with the stored numbers filled in at every pixel.
- A neural fragment shader, a small neural network running on the graphics chip, takes those numbers plus the camera position and outputs a color for each pixel.
Because opacity is on or off, the polygons never need sorting by depth each frame, so they load once. The feature image can also be drawn at double resolution and then shrunk, which averages the numbers before the network runs. That smooths jagged edges while the network still runs only once per final pixel.
A separate claim covers training. Larger neural networks learn the features, the opacity and the mesh shape from photos, by comparing rendered images against the real ones and adjusting. Afterward the results are baked into texture images, and polygons never visible in the photos are pruned away.
This approach enables NeRFs to be rendered with the traditional polygon rasterization pipeline, which provides massive pixel-level parallelism, achieving interactive frame rates on a wide range of compute platforms, including mobile phones.
Translation: This method lets standard graphics hardware run complex 3D scenes smoothly on everyday smartphones.
Why Google's 3D rendering patent could reach phones
If this works as described, lifelike 3D scenes built from photos could open on your phone or in a browser tab instead of needing a gaming PC. The filing lists phones, tablets, smartwatches, home assistants and in-car computers as possible targets. It also says storing surfaces instead of full 3D volumes uses much less memory and power, and that the scenes stay simple triangle meshes you can edit in real time.
The application traces back to a July 2022 provisional filing, and an examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. Nothing here says a product is coming, only that the technique is built from parts nearly every device already has.
That makes this Google's second filing we've tracked in our GPU rendering race watchlist since August, following their work on hardware-free 3D animation.
This patent needs no new hardware. It runs on the graphics chips already inside phones, and the filing's own demo viewer is a plain web page.
What has to exist first is the scene itself. Someone has to capture photos, then run a heavy training process on stronger machines to produce the polygons and textures. The phone only does the last step, playing the result back. The shortest route to a product is a browser viewer for pre-built 3D scans, since the filing already describes one.
The limit is that it models only on-or-off opacity, so see-through things like glass look like a weak spot. Even so, as a software-only playback method, it sits about as close to shippable as a research idea gets.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0340925 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in