Microsoft
Microsoft · AI / ML · Filed Jul 21, 2026 · USPTO published Nov 12, 2026 · This week's release
Microsoft Files Patent for Giving One Big AI Model Many Specialties With Small Add-Ons
Teaching a giant AI model a new job usually means storing and loading a full adjusted copy of the whole thing. Microsoft's patent describes leaving the big model alone and training a tiny add-on instead.
- Publication number
- US 2026/0359642 A1(US20260359642A1)
- Applicant
- Microsoft Technology Licensing, LLC
- Filed · Published
- Jul 21, 2026 · Nov 12, 2026
- Inventors
- Weizhu CHEN, Jingfeng HU, Yelong SHEN, Shean WANG, Yabin LIU
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 706/15
- Parent application
- Continuation of 17/325,117 (filed May 19, 2021)(17325117)
What Microsoft's add-on AI tuning actually does
Imagine you hire someone who has read most of the internet, then ask them to work as a tax specialist on Monday, a customer-support agent on Tuesday, and a database helper on Wednesday. Today, teaching a giant AI model a new specialty often means making a full, adjusted copy of the whole thing, and swapping those copies in and out is slow.
Microsoft's patent takes a lighter route. The big model stays frozen, and each job gets its own tiny set of adjustments. In the filing's example, a single layer has 100 million adjustable numbers, while the add-on for it has only 160,000.
When it's time to work, the adjustments get blended into the model so it runs at the same speed. To change jobs, you peel one add-on off and snap another on.
… merging each frozen base weight matrix with a product of its corresponding pair to form an adapted weight matrix; and performing inference on domain input using the adapted weight matrices.
Translation: The system combines the unchanging core model with the small specialty add ons to run tasks in that specific subject.
How the small add-ons fold into the big model
The claims cover a transformer-based language model (the design behind most modern chatbots) that was first trained on general text. Each of its layers contains a frozen base weight matrix, a big grid of numbers of size d by d that stores what the model learned. Those numbers are never changed.
For each layer, the system obtains a pair of small low-rank adaptation matrices (two slim grids of numbers) already trained on domain-specific data. One is d by r and the other is r by d, where r is much smaller than d. One dependent claim puts r at one percent of d or less. In the filing's example, d is 10,000 and r is 8, and typical values of r run below 100.
The method then merges them: it multiplies the pair together and adds the result to the base grid, producing an adapted weight matrix used for inference (the step where the model answers a new input). A dependent claim says this adds no extra delay. The description also notes the product can be scaled by a factor of α/r.
Switching jobs works in reverse: subtract the product to recover the original grid, then add a second pair trained on different data. The description also says one of the pair starts at zero during training, so the model begins exactly as the original.
The first domain model is trained with first domain specific training data without modifying base model weight matrices.
Translation: The base model learns new subjects by training the small add ons while keeping its original core completely untouched.
Why swapping AI specialties gets cheaper
The practical payoff is for anyone running AI as a service. The filing describes a company keeping one big model loaded and holding potentially hundreds of small modules on stand-by, one per task or even one per customer. Swapping a module is quick, while swapping a whole fine-tuned model means loading a very large file into memory each time.
For you, the benefit would show up indirectly: a bank's assistant, a coding helper and a medical-notes tool could all run on the same underlying model at lower cost. The filing also says the original model's general skills stay intact, since bypassing the add-on falls back to the original. And because the pieces merge before use, answers should arrive at the same speed.
Microsoft's 46th application we've tracked since May on AIs working in teams builds on earlier work like two small AIs sharing tasks and AIs that skip English.
Source: US 2026/0359642 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
Add Patentlyze on Google
One click.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 5 sheets
5 drawing sheets from US 2026/0359642 A1 · click any drawing to enlarge
More from Microsoft
All 646 Microsoft patents →
More in AI training & infrastructure
All 316 AI training & infrastructure patents →
Google
Google Files Patent for Big AI Models That Write Small Programs for Cheaper Ones
Adobe
Adobe Files Patent to Have AI Models Grade Each Other's Answers
Google
Google Patents a System That Picks the Right AI Model for Each Question
Qualcomm
No comments yet