Nvidia Files Patent for Letting Two Chip Types Share Self-Driving Camera Work
Before a self-driving car can recognize a pedestrian, it has to find the corners and edges in every camera frame, and that early chore can eat most of a chip's time. Nvidia's patent splits it between two kinds of cores that work on different frames at once.
What Nvidia's two-chip image trick actually does
Every time a camera on a self-driving car snaps a new frame, a chip has to scan it for corners and edges, the landmarks a car uses to make sense of what is around it. Then the next frame arrives, and the chip has to do it all over again, many times a second.
Nvidia's filing splits that chore between two kinds of processing cores. One reads a picture a single row of pixels at a time. The other grabs a whole square block at once, which suits corner-finding well. Instead of making one core do everything, the patent has them pass work back and forth: the block-reader finds candidate corners, the row-reader handles the prep and bookkeeping, and the block-reader then picks the strongest results.
The overlap is the real trick. While one core finishes up frame one, the other is already starting frame two, so neither sits idle waiting for its turn.
Aspects of this technical solution can increase speed of processing in low-latency application areas, while maintaining integrity of image feature recognition at those higher speeds.
Translation: The method speeds up visual data processing for self-driving cars without losing accuracy.
How the two processors hand work back and forth
The patent covers a two-processor relay. A first processor performs an operation on camera image data. When it finishes, a second processor takes that output plus the original image and performs the next operation. The final result then drives some machine action, such as steering decisions in the vehicle examples.
The two processors are built differently. A pixel processing engine (PPE) works on a two-dimensional grid of pixels at once, like looking at a square tile. A vector processing unit (VPU) works on one row at a time. For comparison, the filing says an ordinary CPU handles a single pixel at a time.
The main example for one frame runs like this:
- The PPE runs Harris corner detection (finding spots where edges meet).
- The VPU runs an injection step on that output.
- The PPE runs non-maximum suppression (keeping only the strongest corner in each neighborhood) while the VPU calculates where the corners sit.
- The VPU then does masking, packing and halo preparation for that frame while the PPE already starts Harris corner on the next frame.
The design also saves work by reusing pixels along tile edges (called halos) instead of loading them again. In one example, a neighborhood check on a 5×5 block of pixels that would need 5 rows and 5 columns loaded on a VPU takes one block and 8 shift instructions on a PPE.
Why faster corner-spotting matters for car cameras
If your car sees the road through cameras, the speed of this early step matters. Every moment spent finding corners and edges is a moment before the car understands what it is looking at. The filing says feature detection on a single core type alone can take up to 80% of the processing time, and that vehicles are adding more cameras at higher resolution. Cutting that cost frees chip time for everything else the car has to do.
The patent reaches beyond cars. Its claims list robots, aerial systems, medical systems and AR headsets among possible uses, though the detailed examples center on in-vehicle cameras. This is chip-level plumbing and no product is named, so read it as a glimpse of how Nvidia thinks about moving camera data efficiently.
Nvidia's 107th filing we've tracked since May in the self-driving sensing race adds to a run that includes one on guessing your turn and one on lane choice without maps.
More cameras at higher resolution mean more pixels to scan every second, and the filing notes that one step alone, finding the edges and corners that let a car recognize a stop sign or a pedestrian, can consume up to 80% of processing time. That leaves almost nothing for the decisions a car actually has to make.
The fix is sized to match the problem. Routing each task to the chip best suited for it, and overlapping one frame's cleanup with the next frame's start, squeezes more throughput from hardware a car already carries.
You will never see this on a screen. If it works as described, it shows up as a car that notices a hazard a fraction of a second sooner, and in a vehicle traveling at highway speed, a fraction of a second is roughly the length of a car.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
21 drawing sheets from US 2026/0350978 A1 · click any drawing to enlarge
Show the other 13 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in