Get the newsletter
Nvidia · AI / ML · Filed Apr 2, 2026 · USPTO published Aug 13, 2026 · Aug 13 release

Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars

A self-driving car that only sees the world in single frozen snapshots will always be a step behind. Nvidia's latest patent filing is about teaching vehicles to layer what they're seeing right now on top of what they saw a moment ago, giving the car a richer mental picture of everything around it.

Side profile of a self-driving car equipped with sensors and cameras along its body. Drawing from patent filing US 2026/0236563 A1.
Side profile of a self-driving car equipped with sensors and cameras along its body.All 14 drawings ↓
Publication number
US 2026/0236563 A1(US20260236563A1)
Applicant
NVIDIA Corporation
Filed · Published
Apr 2, 2026 · Aug 13, 2026
Inventors
Jiwoong Choi, Jose Manuel Alvarez Lopez, Shiyi Lan, Yashar Asgarieh, Zhiding Yu
Status when we published
Waiting for an examiner
Document
20 claims
US classification
382/104
Parent application
Continuation of 18/185,074 (filed Mar 16, 2023)(18185074)

How Nvidia's self-driving cars piece together the road over time

Ever wondered how a self-driving car knows whether that blur at the edge of the frame is a cyclist speeding up or a plastic bag blowing across the road? A single camera frame often can't tell you, but a sequence of frames over time can.

Nvidia's patent describes a system where a vehicle's AI doesn't just look at what its cameras see right now, it stacks that image together with processed snapshots from recent moments, blending them into one combined picture. That combined picture is what the car then uses to decide where to steer, when to brake, and how to plan its route.

The key word here is fusion. Instead of treating each camera frame as a standalone observation, the AI is trained to weave multiple frames across time into a single, more informed view of the world around the vehicle. Think of it like the difference between judging a stranger's mood from one photograph versus watching a few seconds of video.

From the filing · CLAIM 1
generate, using one or more machine learning models and based at least on fused feature maps associated with a plurality of time steps, a temporally fused feature map, the fused feature maps generated using sensor data obtained using the external sensors …

Translation: The system creates a single map by combining visual data from the car's sensors across several different moments in time.

How the model fuses sensor snapshots across multiple time steps

The system uses a machine learning model (MLM) to process raw image data from cameras placed around a vehicle. Each set of camera images is turned into a feature map (essentially a compressed numerical representation of what the cameras see, stripped of raw pixels and focused on meaningful patterns like edges, shapes, and distances).

Critically, the model doesn't just produce one feature map for the current moment. It also holds onto feature maps from previous time steps and runs them through dedicated layers designed to perform a temporal transformation (a mathematical adjustment that accounts for the fact that the car has moved since the earlier frame was captured, so objects need to be repositioned correctly before blending).

Once those historical feature maps are transformed and aligned, they are fused together with the current feature map to create a temporally fused feature map. That single combined map is what drives downstream decisions:

  • Path planning (where should the car go next?)
  • Navigation (is the route still clear?)
  • Vehicle control (when to accelerate, brake, or turn?)

The whole process runs on a combination of CPUs, GPUs, and dedicated hardware accelerators inside the vehicle, suggesting this is designed for real-time, on-board inference rather than cloud-assisted processing.

From the filing · THE ABSTRACT
Systems and methods are disclosed that use a machine learning model (MLM) to intrinsically fuse feature maps associated with different sensors and different instances in time.

Translation: The technology uses artificial intelligence to blend data from various cameras and past observations into one cohesive view.

What this means for autonomous vehicle perception pipelines

For passengers and pedestrians alike, the practical upside is a vehicle that makes fewer mistakes at the edges of its perception, particularly with objects that are partially hidden, moving fast, or only briefly visible. A system that remembers what it saw half a second ago can often figure out what's happening now with much more confidence than one working from a single frozen frame.

Nvidia already supplies the chips and software stack that power many of the leading autonomous and semi-autonomous driving programs, so a patent like this fits squarely into the company's push to own the full perception pipeline, from silicon to the AI model running on it. Autonomous vehicle perception is one of the more active areas across this week's Big Tech patents, and this filing shows how much engineering work still goes into something as apparently simple as "seeing the road clearly."

Source: US 2026/0236563 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 14 sheets

14 drawing sheets from US 2026/0236563 A1 · click any drawing to enlarge

Machine learning model processing pipeline, drawing sheet 1 from US 2026/0236563 A1
Sheet 1 Machine learning model processing pipeline
Processing multiple camera frames, drawing sheet 2 from US 2026/0236563 A1
Sheet 2 Processing multiple camera frames
Fusing feature maps together, drawing sheet 3 from US 2026/0236563 A1
Sheet 3 Fusing feature maps together
Convolutional layers and feature maps, drawing sheet 4 from US 2026/0236563 A1
Sheet 4 Convolutional layers and feature maps
Free-space maps and detected objects, drawing sheet 5 from US 2026/0236563 A1
Sheet 5 Free-space maps and detected objects
Training engine using ground truth data, drawing sheet 6 from US 2026/0236563 A1
Sheet 6 Training engine using ground truth data
Method for processing image feature maps, drawing sheet 7 from US 2026/0236563 A1
Sheet 7 Method for processing image feature maps
Steps for temporal transformation and fusion, drawing sheet 8 from US 2026/0236563 A1
Sheet 8 Steps for temporal transformation and fusion
Show the other 6 sheets
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 9
Sheet 9
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 10
Sheet 10
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 11
Sheet 11
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 12
Sheet 12
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 13
Sheet 13
Nvidia Patents a Way to Merge Past and Present Camera Feeds in Self-Driving Cars patent drawing, sheet 14
Sheet 14
Patent filing page

More in AI recommendation

All 91 AI recommendation patents →

Watchlists this filing belongs to

All watchlists →