Nvidia Files Patent for Loading Only the Part of an AI Model a Task Needs
Nvidia's filing splits one big AI model into specialist sections and loads only the one your photo calls for, so a model too big for a device could still run on it.
What Nvidia's load-only-what-you-need AI does
You point your camera at a tree and ask an app what it is. The AI answering you may be one enormous block that has to sit in memory all at once, even though most of it knows nothing about trees.
Nvidia's patent splits that block into several specialists, say one for plants, one for animals, one for people and one for everyday objects. A small first stage takes a quick look at your photo, noticing things like color and shape. Mostly green? It sends the photo to the plant specialist and loads only that one.
The rest stay on the shelf. The payoff is that a model too big for a device could still run on it, as long as each specialist fits on its own. The filing also applies the idea to reading text, with separate specialists for scanned documents, video, video games and car cameras.
select one or more sub-networks from among a plurality of sub-networks of the one or more neural networks to complete the inferencing based, at least in part, on the input data or the output values from the one or more layers; load the selected one or more sub-networks to the memory …
Translation: The system picks only the specific parts of the AI needed for the job and loads them.
How a quick first look picks the right sub-network
The claim covers a system with memory and processors. Early layers of a neural network (the first stages of the math an AI does) process the input and produce output values. The processors then pick one or more sub-networks from a larger set, based on the original input, those early outputs, or both.
The chosen sub-networks get loaded into memory, and the early outputs are handed to them to finish the job. Those outputs are copied to each chosen sub-network, so the shared early work is done once and reused. The filing calls the picker a router and the shared early layers a stem.
- Stem: cheap early layers that do things like edge detection or build a color histogram (a tally of which colors appear in an image).
- Router: uses quick signals such as color, shape or texture to decide which specialist to activate.
- Sub-networks: the specialists, such as plants, animals, people and objects, or text readers for still images, video, video games and autonomous-vehicle video.
Sub-networks that aren't chosen never get loaded. The filing also says more than one can be activated at a time, and that the specialists can be identical in structure or different. Because only the chosen ones need to fit on the device, the full model can hold an arbitrarily large number of them.
… portions of one or more neural networks are selected for use in inferencing based, at least in part, on information to be inferenced by the one or more neural networks.
Translation: Parts of the AI are chosen based on what the specific task requires.
Why this could fit bigger AI on smaller devices
Memory limits are one reason big AI features often can't run on the device in your hand. Loading only the relevant slice of a model could let larger models run on limited hardware, such as a phone, a laptop or a car's computer, which the filing lists among the places this logic could be used.
The filing's own examples are everyday ones: sorting photos by subject and reading text in scans, video, games and car cameras. On the data center side, loading less per request means less memory and less computing for each answer. It is a patent application, so none of this is a product, and the text we reviewed names no specific one.
Nvidia's 14th filing we've tracked since July on AI models working in teams adds to earlier work like training on mixed image types and routing speech audio.
Memory decides where an AI model can run, and the filing opens by saying so: the memory and computing needs of neural networks can be prohibitively large. That cost is paid in expensive chips, slower answers, and models that simply won't fit on a phone or in a car.
The fix matches the size of the problem. If a photo of a tree only needs the plant specialist, there is no reason to haul the animal and people specialists into memory. Skipping work you never needed is a hard saving to argue with.
The weak spot is the quick first look. A router that judges by color or texture can guess wrong, and a wrong guess sends your photo to the wrong specialist. The filing leans on cheap signals, so how well they hold up is the open question. Even so, the approach is proportionate to a costly problem.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
56 drawing sheets from US 2026/0321048 A1 · click any drawing to enlarge
Show the other 48 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in