Microsoft Files Patent for Cutting AI Training Costs by Trimming Specialist Parts
Some giant AI models already skip most of their own parts to save effort. Microsoft's patent application trims the parts that are left, too.

- Publication number
- US 2026/0368480 A1(US20260368480A1)
- Applicant
- Microsoft Technology Licensing, LLC
- Filed · Published
- Jul 27, 2026 · Nov 19, 2026
- Inventors
- Bita DARVISH ROUHANI, Douglas Christopher BURGER, Eric S CHUNG
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 706/16
- Parent application
- Continuation of 17/657,604 (filed Mar 31, 2022)(17657604)
What Microsoft's double-trimmed AI experts do
Ever hired a team of specialists, only to find you still had to haul in every one of their giant toolboxes? Something like that happens inside many big AI models.
Some large models are built from many small specialist networks, called experts, and only a couple of them work on each word or image patch. That already saves effort. But the experts that do get picked are still full-size, packed with connections that barely matter.
Microsoft's patent application adds a second trim. Each picked expert gets a slimmed-down version, with a fixed share of its connections set to zero. The filing says the two savings multiply, and the combined effect can reach 99% sparsity, meaning almost nothing is left to compute. That means less math, and less to load into memory, every time the model answers you.
… applying coarse-grained sparsity to the mixture of experts layer to reduce a global sparsity level of the mixture of experts layer by designating one or more neural network experts as a subset of the plurality of neural network experts in the mixture of experts layer to train on each input data shard …
Translation: The system selects only a small group of specialist neural networks to work on each piece of data.
How the trimmed experts are picked and trained
Claim 1 covers a way of training a model built from "mixture of experts" layers. A routing gate network (a small judge that decides who handles what) receives pieces of input, such as image patches or word fragments called tokens, and sends each piece to just a few of the experts.
That first step is coarse-grained sparsity: most experts sit idle for any given piece. Pick 2 experts out of 10, the description says, and 80% of the layer is skipped. The second step is fine-grained sparsity: each chosen expert uses a stored weight matrix (the grid of numbers holding what it has learned) with a preset share of entries zeroed out, so it has fewer parameters than the full expert.
The model then runs the piece through that slimmed expert to make a prediction, checks the error, works backward through the network (backpropagation), and updates the slimmed grid directly.
The dependent claims add options:
- Balanced "N of M" sparsity: only N of every M values may be non-zero, with blocks of at least 8, so chips can predict the pattern.
- Sparsity tied to routing: the amount of trimming can follow the ratio of chosen experts to total experts, or the layer's depth.
In the description's example, three layers are trimmed 75%, 62.5% and 50%, with the most specialized experts trimmed least. Another independent claim has a storage device trim a stored matrix on request and hand it to the layer.
For each designated neural network expert, a weight matrix is retrieved having a predetermined sparsity to generate a sparsified designated neural network expert.
Translation: The chosen specialists are then trimmed down to use fewer parameters and run more efficiently.
Why trimmed AI models could cost less to run
Running giant AI models is expensive, and a lot of the cost is moving numbers around. The filing says that when a model answers a question, loading an expert's weights takes more time than the multiplication itself, so smaller experts load faster and need less math. If the approach works as described, it points toward cheaper training and quicker answers from the AI services you already use.
The description also admits a weakness of today's expert-based models: when answering questions, they can use under 5% of the available computing power because the work is spread unevenly. Microsoft's answer here is mostly to make each piece lighter, and it pairs the idea with chips built for the N of M pattern. The benefit stays on paper until someone tests it at scale.
That makes this Microsoft's 50th filing we've tracked since May in our AI teams working together watchlist, following applications like the self-correcting safety rewriter and the self-grading ad coach.
Source: US 2026/0368480 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 11 sheets
11 drawing sheets from US 2026/0368480 A1 · click any drawing to enlarge

No comments yet