Nvidia Patents a Multi-Sensor Location System for Self-Driving Vehicles

By Patentlyze Team · Updated Jun 26, 2026

Getting a self-driving car to know exactly where it is on the road is harder than it sounds. Nvidia's latest patent describes a system that cross-checks data from multiple sensors at once to produce a more confident, reliable position fix.

How Nvidia's self-driving location system actually works

Imagine you're trying to figure out where you are in an unfamiliar city using both a street-level photo app and a radar sweep at the same time. Instead of trusting just one source, you compare what both tools see against a detailed map, then combine those two answers into one confident location. That's essentially what this patent describes for self-driving vehicles.

Nvidia's system collects sensor data from cameras, lidar, radar, or any mix of sensors on a moving vehicle, then matches what each sensor sees against a pre-built high-definition (HD) map. That map was itself created by pooling data from many vehicles driving the same roads over time, all uploaded to the cloud and stitched together.

The key move is that each sensor type produces its own location estimate, and then those estimates are combined, or "fused," into a single final answer. If one sensor is confused by glare or rain, the others can compensate. The result feeds directly into the vehicle's planning and navigation decisions.

How sensor fusion and HD map matching lock in position

The patent covers three linked stages: data generation, map creation, and real-time localization.

In the first stage, vehicles generate what the patent calls mapstreams, bundles of raw sensor data, outputs from deep neural networks (AI models that identify objects and lane markings), and movement trajectory logs. These are uploaded to the cloud from any number of vehicles making any number of trips.

• Map creation: The cloud aggregates all those mapstreams into a single fused HD map that reflects what the road looks like across many drives and conditions.
• Per-modality localization: When a vehicle is driving, it runs a separate location check for each sensor type (camera, lidar, radar, etc.) by comparing live sensor data to the slice of the HD map that was built from that same sensor type.
• Fusion: The individual location results from each sensor are then merged into one final position estimate, weighted by how confident each sensor's match was.

The claim covers an autonomous or semi-autonomous machine with CPUs, GPUs, and dedicated hardware accelerators handling this pipeline, with the final location feeding directly into planning, navigation, and control operations.

What this means for autonomous vehicle reliability

Self-driving vehicles fail most often when a single sensor is degraded, a camera blinded by sun, a lidar confused by heavy rain. A system that runs independent location checks for each sensor and then fuses the results is fundamentally more fault-tolerant than one that relies on a single data stream. For Nvidia, whose DRIVE platform already powers vehicles from dozens of automakers, this kind of localization infrastructure is core to the value proposition.

The cloud-based map creation piece is also strategically significant. It means each vehicle on the road is contributing to a shared map that makes every other vehicle more accurate. The more vehicles Nvidia's platform powers, the better that shared map gets, which is a compounding advantage over time.

Editorial commentary on a publicly published patent application. Not legal advice.