Nvidia Patents a Self-Driving System That Uses Two Decision-Making Systems at Once
Most self-driving systems pick one approach and commit to it. Nvidia's new patent describes a system that runs two different planning methods side by side and picks the better answer every single time.
How Nvidia's two-planner driving system actually works
Every time a self-driving car has to decide whether to change lanes, two broad philosophies are fighting for the wheel: old-school rules written by human engineers, and newer AI that learned to drive by watching data. Neither one is perfect on its own.
Nvidia's patent describes a system that runs both at the same time. One path follows hand-crafted rules to map out a route. A separate AI path generates its own independent suggestion. The car then picks which suggestion to actually use, depending on where it is and what conditions it's facing.
The clever part is that Nvidia designed this so both methods can share the same onboard chips without one starving the other of processing power. That makes it a practical bridge between today's partially automated cars and a future where the AI handles everything.
generate, based at least on applying a first representation of sensor data obtained using one or more sensors of an ego-machine to a first signal path comprising a first encoder network and a heuristic- or rules-based planner, a first candidate action …
Translation: The system uses traditional rules to calculate a driving move from sensor data.
How the hybrid stack picks between two candidate actions
The patent describes what Nvidia calls a hybrid drive stack: two separate signal paths that each take in sensor data (cameras, radar, lidar) and each produce a candidate driving action independently.
The first path uses a classical encoder and heuristic planner (think: explicit rules like "keep three seconds of following distance" or "don't cross a solid white line"). It processes the sensor data and outputs a candidate trajectory. The second path uses a neural encoder and neural planner, an end-to-end AI model trained on real-world driving data, which outputs its own separate candidate trajectory.
A selection mechanism then decides which candidate to send to the vehicle controls. The patent notes that selection can depend on the navigational domain (for example, a familiar highway stretch versus an unusual intersection), so the system can favor the AI planner when it's confident and fall back to the rules-based planner when it isn't.
- Both paths have their own encoder (feature extractor), so neither shares learned representations that could bias the other.
- Either or both encoders can be scaled down in complexity to fit on the same system-on-chip without overloading it.
- The architecture is designed to let teams gradually migrate from a fully rules-based stack to a fully AI-driven one, rather than requiring a sudden switchover.
A hybrid control stack may be implemented with classical and E2E signal paths with independent feature extractors or backbone networks.
Translation: The vehicle combines old school programming with end to end neural networks.
What this means for the road to fully autonomous vehicles
For anyone who rides in or buys a vehicle with driver-assistance features, this kind of architecture could mean fewer edge-case failures. Rules-based systems handle well-understood situations reliably; AI planners handle messy, ambiguous ones better. A car that can lean on each strength at the right moment is, in principle, safer than one that bets everything on one approach.
For the industry, Nvidia's long bet on autonomous-vehicle computing makes this filing notable. It shows Nvidia is not just selling chips to robotaxi companies but building the software architecture that sits on top of those chips. Whoever controls the planning stack shapes how autonomy actually develops, not just how fast it runs.
Nvidia's 67th filing we've tracked since May in the self-driving sensing race adds to a run that includes a two-memory simulation system and a stitched video camera system.
The design has a real cost that the patent doesn't fully resolve: two independent encoders processing the same sensor data means two sets of computations, two sets of potential errors, and two representations that could disagree in ways that make the selection logic harder than it looks. Deciding which planner to trust, and when, is itself a hard problem that the patent largely defers to "any suitable technique."
That caveat does a lot of work. The value of the hybrid stack depends almost entirely on how well the selection layer works, and if that layer uses yet another learned model to pick between the two planners, you've just pushed the reliability question up one level.
Still, the idea of a gradual migration path from rules-based to AI-driven control is a practical concession to reality. Full end-to-end AI planners are hard to certify for safety because their decisions are difficult to explain to regulators. Keeping a rules-based path running in parallel gives engineers a fallback they can actually audit. That tradeoff reads as worth it for where the industry is right now.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
14 drawing sheets from US 2026/0274298 A1 · click any drawing to enlarge
Show the other 6 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in