Nvidia Patents a Neural Network That Reconstructs Missing Medical Scans

By Patentlyze Team · Updated May 29, 2026

What if a hospital only has a patient's MRI but needs a CT scan to make a treatment decision? Nvidia is patenting an AI system that can synthesize the missing scan from the ones that already exist.

How Nvidia's AI fills in a missing MRI or CT scan

Imagine a cancer patient arrives at a clinic that only ran one type of scan — say, an MRI — but the radiologist really needs a CT scan or a PET image to make a confident diagnosis. Getting another scan takes time, money, and sometimes exposes the patient to more radiation. That's a real, everyday problem in medical imaging.

Nvidia's patent describes a neural network approach that looks at the scans you do have and uses them to generate the ones you don't. The core insight is that different types of scans of the same body part share a lot of information — the shape of a tumor, for instance — but each modality also captures something unique, like how tissue absorbs contrast dye differently in MRI versus CT.

By teaching the AI to separate the shared information (common features across all scan types) from the unique information (what only that scan type captures), the system can plausibly reconstruct a missing scan type with far greater accuracy than a brute-force approach.

How the GAN disentangles shared vs. unique scan features

The patent describes a generative adversarial network (GAN) — a two-part AI setup where one network generates synthetic images and another critiques them, pushing quality higher through competition — trained specifically for multi-modality medical image completion.

The key technical move is called representational disentanglement. When you feed the system a set of related scans (say, T1-weighted and T2-weighted MRI sequences, or MRI plus CT), it decomposes each scan's internal representation into two buckets:

• Common components — features shared across all modalities, like the underlying anatomy and pathology structure
• Unique components — features specific to one scan type, like the contrast response or tissue signal characteristics of that particular imaging method

By encoding scans this way, the GAN can recombine the shared anatomical knowledge with the learned "style" of a missing modality to synthesize a plausible substitute scan. The training process uses the complete multi-scan datasets (when all modalities are available) to teach the model what each modality's unique fingerprint looks like, so it can reconstruct missing ones at inference time.

The inventors — who span Nvidia's medical imaging research team — are applying this to scenarios where one or more scan types in a standard clinical protocol are missing, incomplete, or too low quality to use.

What this means for hospitals with incomplete imaging data

In clinical practice, incomplete imaging datasets are surprisingly common — patients can't tolerate long scan times, equipment isn't available, or certain sequences were skipped. AI-driven image synthesis could allow downstream diagnostic models (tumor segmentation, treatment planning tools) to keep running even when data is missing, rather than failing or producing degraded results.

For Nvidia, this fits squarely into its Clara medical AI platform strategy. Hospitals running Nvidia-powered diagnostic pipelines would benefit from a built-in fallback that patches gaps in patient data. It also has research value: synthesizing rare or expensive scan types from cheaper, more common ones could democratize access to multi-modal diagnostics in under-resourced healthcare settings.

Editorial commentary on a publicly published patent application. Not legal advice.