Nvidia Patents a Two-Stage Image Sorter That Trains Self-Driving Cars More Efficiently

By Patentlyze Team · Updated Jul 3, 2026

Teaching a self-driving car to recognize a pedestrian requires thousands of labeled example images. Nvidia's new patent describes a system that finds and labels those images automatically, without a human sorting through footage frame by frame.

How Nvidia's self-driving AI learns from its own data searches

Imagine trying to teach someone to recognize a stop sign by showing them millions of dashcam photos, most of which contain no stop signs at all. Someone still has to find the useful photos and label them. That sorting work is tedious, expensive, and slow, and it has to happen before any AI training can even begin.

Nvidia's patent describes a system that does this sorting in two automated passes. The first pass scans a large pile of images and flags the ones that probably contain a relevant object (say, a pedestrian or a traffic cone). The second pass looks only at those flagged images and assigns them a precise label. Each round of sorting makes the first filter a little more accurate, so the system improves over time without extra human effort.

The end result is a labeled dataset that can be fed directly into the AI models that control a self-driving vehicle's steering, braking, and navigation. Less manual work, faster iteration, and more data to train on.

How the two classifiers filter and label images in sequence

The patent describes a data-mining pipeline built around two classifiers working in sequence. A classifier here is simply an AI model trained to sort things into categories, like a spam filter deciding which emails are junk.

• First classifier (group filter): takes a large, unsorted set of images and identifies which ones belong to a broad category of interest, for example, images that contain a vehicle, a person, or any object in a defined group. It outputs a smaller, cleaner subset.
• Second classifier (object labeler): receives that smaller subset and assigns specific labels to the objects it finds, distinguishing a cyclist from a pedestrian, or a parked car from a moving one.
• Iterative refinement: after each pass, the first classifier is updated using the results, so it gets better at spotting relevant images in the next round.

The labeled images that survive both filters are stored as training data for the navigation and control models that actually drive the vehicle. Those models use the curated data to learn planning and control decisions, connecting the data-mining pipeline directly to real-world autonomous driving behavior.

The system is designed to scale: running more iterations produces more accurate filters and more useful training sets without proportionally more human annotation work.

What this means for the cost of building self-driving AI

Building training data is one of the most expensive parts of developing any AI system, and for autonomous vehicles it is especially painful because the relevant objects (a child running into the road, a broken-down truck half in a lane) appear rarely in raw footage. A system that can automatically mine for those rare cases and label them reduces both the cost and the time it takes to improve a self-driving model.

For Nvidia, which sells the compute platforms that power many autonomous vehicle programs, this kind of automated data pipeline also makes its hardware more attractive. If the software toolchain that runs on Nvidia chips can generate its own training data, customers spend less time on manual annotation work and more time training models on Nvidia GPUs.

Editorial commentary on a publicly published patent application. Not legal advice.