Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars
A self-driving car that only sees the world in single frozen snapshots will always be a step behind. Nvidia's latest patent filing is about teaching vehicles to layer what they're seeing right now on top of what they saw a moment ago, giving the car a richer mental picture of everything around it.

- Publication number
- US 2026/0236563 A1(US20260236563A1)
- Applicant
- NVIDIA Corporation
- Filed · Published
- Apr 2, 2026 · Aug 13, 2026
- Inventors
- Jiwoong Choi, Jose Manuel Alvarez Lopez, Shiyi Lan, Yashar Asgarieh, Zhiding Yu
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 382/104
- Parent application
- Continuation of 18/185,074 (filed Mar 16, 2023)(18185074)
How Nvidia's self-driving cars piece together the road over time
Ever wondered how a self-driving car knows whether that blur at the edge of the frame is a cyclist speeding up or a plastic bag blowing across the road? A single camera frame often can't tell you, but a sequence of frames over time can.
Nvidia's patent describes a system where a vehicle's AI doesn't just look at what its cameras see right now, it stacks that image together with processed snapshots from recent moments, blending them into one combined picture. That combined picture is what the car then uses to decide where to steer, when to brake, and how to plan its route.
The key word here is fusion. Instead of treating each camera frame as a standalone observation, the AI is trained to weave multiple frames across time into a single, more informed view of the world around the vehicle. Think of it like the difference between judging a stranger's mood from one photograph versus watching a few seconds of video.
generate, using one or more machine learning models and based at least on fused feature maps associated with a plurality of time steps, a temporally fused feature map, the fused feature maps generated using sensor data obtained using the external sensors …
Translation: The system creates a single map by combining visual data from the car's sensors across several different moments in time.
How the model fuses sensor snapshots across multiple time steps
The system uses a machine learning model (MLM) to process raw image data from cameras placed around a vehicle. Each set of camera images is turned into a feature map (essentially a compressed numerical representation of what the cameras see, stripped of raw pixels and focused on meaningful patterns like edges, shapes, and distances).
Critically, the model doesn't just produce one feature map for the current moment. It also holds onto feature maps from previous time steps and runs them through dedicated layers designed to perform a temporal transformation (a mathematical adjustment that accounts for the fact that the car has moved since the earlier frame was captured, so objects need to be repositioned correctly before blending).
Once those historical feature maps are transformed and aligned, they are fused together with the current feature map to create a temporally fused feature map. That single combined map is what drives downstream decisions:
- Path planning (where should the car go next?)
- Navigation (is the route still clear?)
- Vehicle control (when to accelerate, brake, or turn?)
The whole process runs on a combination of CPUs, GPUs, and dedicated hardware accelerators inside the vehicle, suggesting this is designed for real-time, on-board inference rather than cloud-assisted processing.
Systems and methods are disclosed that use a machine learning model (MLM) to intrinsically fuse feature maps associated with different sensors and different instances in time.
Translation: The technology uses artificial intelligence to blend data from various cameras and past observations into one cohesive view.
What this means for autonomous vehicle perception pipelines
For passengers and pedestrians alike, the practical upside is a vehicle that makes fewer mistakes at the edges of its perception, particularly with objects that are partially hidden, moving fast, or only briefly visible. A system that remembers what it saw half a second ago can often figure out what's happening now with much more confidence than one working from a single frozen frame.
Nvidia already supplies the chips and software stack that power many of the leading autonomous and semi-autonomous driving programs, so a patent like this fits squarely into the company's push to own the full perception pipeline, from silicon to the AI model running on it. Autonomous vehicle perception is one of the more active areas across this week's Big Tech patents, and this filing shows how much engineering work still goes into something as apparently simple as "seeing the road clearly."
Source: US 2026/0236563 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.
The drawings · 14 sheets
14 drawing sheets from US 2026/0236563 A1 · click any drawing to enlarge
