Nvidia Patent Teaches AI to Reconstruct Hidden Portions of Occluded Objects
When a person steps partially behind a lamppost, you still know it's a whole person. Teaching computers to do the same thing, reliably, turns out to be one of the harder problems in machine vision. Nvidia just filed a patent on a clever shortcut.
How Nvidia's AI guesses what's behind an obstruction
You're watching a parking lot camera feed, and a shopping cart is half-hidden behind a parked van. A standard AI sees the visible half and treats that as the whole object. It has no idea what shape lies behind the van, which means it can badly misjudge where the cart is or how big it is.
Nvidia's patent describes a training method that teaches an AI to mentally "complete" objects, filling in the parts it can't actually see. The system learns this by creating its own practice problems: it takes images of unobstructed objects, digitally places blockers in front of them to simulate occlusion, and then trains the AI to work backwards from the partially hidden version to the full original shape.
The result is a model that, given a photo and a rough box drawn around an object, can output a mask showing the complete outline of that object, even the hidden portions. This kind of perception is sometimes called "amodal" vision, meaning it goes beyond what the camera literally captures.
… generating, based on the first predicted amodal mask, the one or more first images, the one or more first bounding box prompts, and one or more ground truth modal masks, unoccluded object data …
Translation: It creates clear object data by combining initial predictions, source images, and ground truth details.
How the two-model training loop builds synthetic occlusions
The patent lays out a two-stage training pipeline built around generating its own labeled data, since real-world images of objects with verified hidden portions are extremely rare and expensive to annotate.
Stage one uses an existing, already-trained model to produce "amodal masks" (pixel maps showing the estimated full shape of an object, including occluded areas) from ordinary photos. Those predictions are then cross-checked against ground truth modal masks (annotations showing only the visible portion of an object, which are much easier to collect). Where the amodal prediction extends beyond the visible mask, the system treats that extension as the estimated hidden region.
Stage two takes those estimated hidden regions and builds a synthetic training set. The system cuts unoccluded objects out of images, composites them into new scenes with artificial blockers in front of them, and records exactly how much of each object is now hidden. This gives the second, student model pairs of: "here is the partially hidden object" and "here is its true full shape."
- Input: a photo plus a bounding box drawn around the object of interest
- Processing: the trained model infers the full object shape from visible pixels and contextual cues
- Output: an amodal mask covering both visible and estimated hidden portions
The clever part is that the pipeline is largely self-bootstrapping. It does not require humans to draw masks around hidden regions, which would be guesswork anyway.
… generating, based on the unoccluded object data, one or more synthetic images and occluded object data …
Translation: It then builds synthetic images and occluded object details using that clean data.
What full-object detection means for robots and self-driving cars
For robotics and autonomous vehicles, knowing the full extent of an object, not just its visible face, is the difference between a safe plan and a dangerous one. A robot arm reaching past a partially hidden bottle needs to know where the whole bottle is. A self-driving car merging past a partially obscured pedestrian needs the full body outline, not just a shoulder. Nvidia's push into autonomous-system AI makes this kind of perception research a natural priority.
For anyone using AI-powered video analysis, retail tracking, or warehouse automation, this approach also matters because it reduces the need for enormous hand-labeled datasets. Generating synthetic occlusions programmatically is much cheaper than paying humans to annotate thousands of real-world partial-occlusion scenarios.
Nvidia's 51st filing we've tracked since May in the self-driving sensing race adds to a run that includes one stitching camera views overhead and a two-stage object spotter.
The pipeline trains its second model using guesses made by a first model about the hidden portions of objects. Any mistake the first model makes gets written into the training data as settled truth, which is a real cost. Nvidia adds a check against human-drawn outlines of visible areas, which catches cases where the first model invents impossible hidden regions, but leaves the opposite problem untouched: a first model that underestimates how much of an object is hidden will teach the second model to do the same.
That asymmetric blind spot is the honest price of this design.
The trade reads as acceptable for real products. Occasional shrinking of guesses about hidden regions matters far less than having a model trained on millions of images rather than hundreds, and the visible-region check at least prevents the wilder errors from compounding into something worse.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0260359 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →