Get the newsletter
Microsoft · AI / ML · Filed Jul 27, 2026 · USPTO published Nov 19, 2026 · This week's release

Microsoft Files Patent for Cutting AI Training Costs by Trimming Specialist Parts

Some giant AI models already skip most of their own parts to save effort. Microsoft's patent application trims the parts that are left, too.

An AI model that switches off unused paths to save computer power while learning new tasks. Drawing from patent filing US 2026/0368480 A1.
An AI model that switches off unused paths to save computer power while learning new tasks.All 11 drawings ↓
Publication number
US 2026/0368480 A1(US20260368480A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Jul 27, 2026 · Nov 19, 2026
Inventors
Bita DARVISH ROUHANI, Douglas Christopher BURGER, Eric S CHUNG
Status when we published
Waiting for an examiner
Document
20 claims
US classification
706/16
Parent application
Continuation of 17/657,604 (filed Mar 31, 2022)(17657604)

What Microsoft's double-trimmed AI experts do

Ever hired a team of specialists, only to find you still had to haul in every one of their giant toolboxes? Something like that happens inside many big AI models.

Some large models are built from many small specialist networks, called experts, and only a couple of them work on each word or image patch. That already saves effort. But the experts that do get picked are still full-size, packed with connections that barely matter.

Microsoft's patent application adds a second trim. Each picked expert gets a slimmed-down version, with a fixed share of its connections set to zero. The filing says the two savings multiply, and the combined effect can reach 99% sparsity, meaning almost nothing is left to compute. That means less math, and less to load into memory, every time the model answers you.

From the filing · CLAIM 1
… applying coarse-grained sparsity to the mixture of experts layer to reduce a global sparsity level of the mixture of experts layer by designating one or more neural network experts as a subset of the plurality of neural network experts in the mixture of experts layer to train on each input data shard …

Translation: The system selects only a small group of specialist neural networks to work on each piece of data.

How the trimmed experts are picked and trained

Claim 1 covers a way of training a model built from "mixture of experts" layers. A routing gate network (a small judge that decides who handles what) receives pieces of input, such as image patches or word fragments called tokens, and sends each piece to just a few of the experts.

That first step is coarse-grained sparsity: most experts sit idle for any given piece. Pick 2 experts out of 10, the description says, and 80% of the layer is skipped. The second step is fine-grained sparsity: each chosen expert uses a stored weight matrix (the grid of numbers holding what it has learned) with a preset share of entries zeroed out, so it has fewer parameters than the full expert.

The model then runs the piece through that slimmed expert to make a prediction, checks the error, works backward through the network (backpropagation), and updates the slimmed grid directly.

The dependent claims add options:

  • Balanced "N of M" sparsity: only N of every M values may be non-zero, with blocks of at least 8, so chips can predict the pattern.
  • Sparsity tied to routing: the amount of trimming can follow the ratio of chosen experts to total experts, or the layer's depth.

In the description's example, three layers are trimmed 75%, 62.5% and 50%, with the most specialized experts trimmed least. Another independent claim has a storage device trim a stored matrix on request and hand it to the layer.

From the filing · THE ABSTRACT
For each designated neural network expert, a weight matrix is retrieved having a predetermined sparsity to generate a sparsified designated neural network expert.

Translation: The chosen specialists are then trimmed down to use fewer parameters and run more efficiently.

Why trimmed AI models could cost less to run

Running giant AI models is expensive, and a lot of the cost is moving numbers around. The filing says that when a model answers a question, loading an expert's weights takes more time than the multiplication itself, so smaller experts load faster and need less math. If the approach works as described, it points toward cheaper training and quicker answers from the AI services you already use.

The description also admits a weakness of today's expert-based models: when answering questions, they can use under 5% of the available computing power because the work is spread unevenly. Microsoft's answer here is mostly to make each piece lighter, and it pairs the idea with chips built for the N of M pattern. The benefit stays on paper until someone tests it at scale.

That makes this Microsoft's 50th filing we've tracked since May in our AI teams working together watchlist, following applications like the self-correcting safety rewriter and the self-grading ad coach.

Source: US 2026/0368480 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 11 sheets

11 drawing sheets from US 2026/0368480 A1 · click any drawing to enlarge

How the AI routes data to specialized virtual workers, drawing sheet 1 from US 2026/0368480 A1
Sheet 1 How the AI routes data to specialized virtual workers
A fully connected network passing data back and forth, drawing sheet 2 from US 2026/0368480 A1
Sheet 2 A fully connected network passing data back and forth
Dropping unused connections to make the network lighter, drawing sheet 3 from US 2026/0368480 A1
Sheet 3 Dropping unused connections to make the network lighter
Two patterns for silencing parts of a network, drawing sheet 4 from US 2026/0368480 A1
Sheet 4 Two patterns for silencing parts of a network
Steps for picking and slimming down specialist AI workers, drawing sheet 5 from US 2026/0368480 A1
Sheet 5 Steps for picking and slimming down specialist AI workers
Trimming more connections as the AI parts gain expertise, drawing sheet 6 from US 2026/0368480 A1
Sheet 6 Trimming more connections as the AI parts gain expertise
Predicting which specialists will be needed next, drawing sheet 7 from US 2026/0368480 A1
Sheet 7 Predicting which specialists will be needed next
Fetching upcoming specialist data ahead of time, drawing sheet 8 from US 2026/0368480 A1
Sheet 8 Fetching upcoming specialist data ahead of time
Show the other 3 sheets
How the AI groups data to load specialist parts, drawing sheet 9 from US 2026/0368480 A1
Sheet 9 How the AI groups data to load specialist parts
Sorting incoming tasks to match them with specialist tools, drawing sheet 10 from US 2026/0368480 A1
Sheet 10 Sorting incoming tasks to match them with specialist tools
Computer hardware that trains and runs the efficient AI software, drawing sheet 11 from US 2026/0368480 A1
Sheet 11 Computer hardware that trains and runs the efficient AI software
Patent filing page

More in AI training & infrastructure

All 323 AI training & infrastructure patents →

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.