Nvidia Patents a LiDAR-Only System That Tracks Moving Objects for Self-Driving Cars

By Patentlyze Team · Updated Jun 4, 2026

Most self-driving perception systems lean heavily on cameras and radar to track how fast other objects are moving. Nvidia's new patent describes a way to do that job with LiDAR data alone — no camera required.

How Nvidia's self-driving cars clock moving objects

Imagine a self-driving car trying to figure out whether that blurry shape ahead is a parked truck or one that's slowly pulling into traffic. Getting that wrong — even for a fraction of a second — is dangerous. Knowing an object's velocity, not just its position, is what separates a safe reaction from a bad one.

Nvidia's patent describes a system that uses LiDAR (the laser-pulse sensor that paints a 3D picture of the world around a vehicle) to continuously track both where objects are and how fast they're moving. The system compares how an object's outline has shifted between laser scans to estimate its speed, then checks whether that moving object is the same one it was already watching.

The key insight is doing all of this with LiDAR alone, without needing to fuse in camera or radar data. That makes the tracker more self-contained and potentially more reliable in conditions where cameras struggle — like glare, darkness, or heavy rain.

How ICP alignment and Kalman filtering lock onto objects

The system works in two interlocking stages: object alignment and state tracking.

For alignment, Nvidia uses an algorithm called Iterative Closest Point (ICP) — a technique that takes two 3D point clouds (the LiDAR scan from one moment and the scan from the next) and figures out how much an object's boundary has shifted. That shift, divided by the time between scans, gives you a measured velocity for that object.

For tracking, the system uses a Kalman filter — a mathematical tool that blends noisy real-world measurements with a model of how objects are expected to behave over time (think: a car doesn't teleport, it moves smoothly). The filter maintains a tracked velocity state for each known object, updating it as new measurements come in.

The glue between the two stages is an association cost — essentially a score that asks: "Does this newly detected object's velocity match what I'd expect from the object I was already tracking?" If the score clears a threshold, the system links the new detection to the old tracked object and updates its state. If not, it may flag it as a new object entirely.

The output feeds directly into the vehicle's planning and navigation stack — so decisions about braking, steering, or yielding are grounded in continuously updated velocity data.

What this means for camera-free autonomous driving

For autonomous vehicles, velocity tracking is arguably more safety-critical than position tracking. A pedestrian who is stationary requires a very different response than one who is stepping into the road at 4 mph. Getting that right in real time — and doing it reliably in low-visibility conditions — is one of the harder unsolved problems in AV perception.

By anchoring the entire pipeline to LiDAR-only data, Nvidia's approach sidesteps some of the sensor-fusion complexity that can introduce latency or failure modes when cameras or radar inputs are degraded. This is likely relevant to Nvidia's DRIVE platform, which powers a wide range of third-party autonomous vehicle programs. A tighter, more self-contained object tracker could make the platform more attractive to OEMs who want robust perception without over-relying on any single sensor modality.

Editorial commentary on a publicly published patent application. Not legal advice.