Nvidia Files Patent for Counting Each Object Once When Several Car Cameras See It
When one truck shows up in three different car cameras, a self-driving system has to work out that it is still just one truck, and do it without a delay. Nvidia's new patent application covers a way to do that quickly.
What Nvidia's multi-camera patent actually does
A delivery van overtakes a self-driving car on the highway. First the rear camera sees it, then the side camera, then the front one, and for a moment two cameras are looking at it at once.
If the car treats each camera's sighting as a separate vehicle, its picture of the road fills up with phantom vans. Nvidia's patent describes software that works out which sightings are really the same object, gives that object a single ID, and hands the car's driving software one combined report. Where two cameras disagree a little on how fast the van is going or exactly where it sits, the system averages their guesses, using a few recent frames to steady the numbers.
The pitch is speed. Older approaches, the filing says, wait for several frames of video to confirm something is real before reporting it. This one reports quickly, which gives your car more time to react.
… identifying at least one subset of the plurality of input objects, the at least one subset corresponding to a respective physical object and comprising at least one input object that satisfies a similarity criterion …
Translation: The system groups together detections from different cameras that likely represent the exact same real world object.
How the fusion engine matches camera sightings
Each camera feeds its own perception pipeline, a neural network (software trained to recognize things in images) that lists what it spots, along with each object's shape, position, speed and acceleration. Those lists are the 'input objects.'
A fusion engine then decides which input objects from different cameras are the same physical thing. It builds a graph (a web of dots joined by lines) where each dot is a detection, each camera gets its own group of dots, and each line carries a 'cost' showing how different two detections look. It picks the set of matches with the lowest total cost. Placeholder 'dummy nodes' stand in for cameras that cannot see an object, so the math works for any number of cameras. The filing says older matching methods were limited to two.
A tracking engine gives each physical object one global ID, keeps a table of what each camera calls it, and stores a short history, such as the last 5 to 10 frames. Then:
- Each camera gets a priority, for example a long-range camera ranks higher for distant objects. If one camera clearly outranks the rest, its report is used as is.
- Otherwise, position, speed and orientation are averaged across cameras and recent frames.
- Detections near the edge of an image get less weight, and their speed and orientation can be dropped, since partly cut-off objects produce errors.
The single combined object goes to the vehicle's planning subsystem.
… generating an output object associated with one or more smoothed object characteristics, the output object being generated based at least on two or more input objects included in the at least one subset …
Translation: It combines the overlapping camera data into a single unified object record to prevent double counting.
Why faster object tracking matters for your car
Driver-assist and self-driving cars make decisions from a running list of what is around them. If that list is slow to update, or counts one van twice, braking and lane choices get worse. Nvidia's filing says its approach cuts the delay by not waiting for several frames of video to confirm each object, which could give a car more time to react.
The filing describes cars in detail, but it also lists robots, drones, warehouse vehicles and boats as places the idea could go. Keep in mind this is a patent application, and it does not say any carmaker or product uses it.
Nvidia's 108th filing we've tracked since May in the self-driving sensing race adds to a run that includes sharing camera work across chips and predicting driver turns.
Claim 1 is short and wide. It asks for several cameras, a way to group detections that look similar, one output object with 'smoothed' traits for anything two or more cameras can see, and delivery of that object to a part of a machine.
Notice what is missing. The claim never says car, never mentions the graph-matching math, never says 'average,' and says nothing about camera priority or image edges. Those details live in the description. If granted as written, the wording would reach any machine that merges duplicate camera sightings and smooths the result, from cars to warehouse robots.
Merging duplicate sightings is a basic need for any multi-camera system, so a claim this open invites close scrutiny. The narrower ideas in the description, such as weighting by camera strength and ignoring speed readings at the edge of the frame, are the more distinctive part.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
14 drawing sheets from US 2026/0350965 A1 · click any drawing to enlarge
Show the other 6 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in