Microsoft Patents a Way to Compress AI Models Without Losing What Matters Most
Running a powerful AI model on a phone or laptop usually means accepting a watered-down version. Microsoft's new patent tries to change that math by being surgical about what gets compressed and what doesn't.

- Publication number
- US 2026/0300700 A1(US20260300700A1)
- Applicant
- Microsoft Technology Licensing, LLC
- Filed · Published
- Mar 31, 2025 · Oct 1, 2026
- Inventors
- Yehonathan REFAEL KALIM, Adam HAKIM, Gilad KIRSHENBOIM
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 706/25
How Microsoft shrinks AI to fit on everyday hardware
You're trying to run an AI assistant directly on your phone, no cloud required, but the model is simply too large for the hardware to handle at full quality. Something has to give.
Microsoft's patent describes a technique for deciding which parts of an AI model are worth preserving in high fidelity and which parts can be safely squeezed down without the model noticing. Think of it like digitizing an old photograph: you keep the subject's face sharp and let the background go a little blurry. The AI keeps its important "neurons" at full precision and compresses the rest aggressively.
The result is a model that takes up far less memory and computing power, making it practical to run on a laptop, phone, or other edge device that can't tap a data center. The goal is to preserve accuracy where it counts while making the tradeoffs in places the model can absorb.
… identifying a first plurality of significant columns and/or rows in the orthogonal matrix that make above a threshold contribution to producing an accurate final inference …
Translation: It finds the parts of the AI that actually matter for making correct guesses.
How the patent separates critical AI weights from disposable ones
The patent centers on a mathematical technique called Singular Value Decomposition (SVD), a method for breaking a large table of numbers (called a matrix) into simpler pieces ranked by how much they contribute to the matrix's overall meaning. In an AI model, each layer of the network contains such matrices, and SVD lets you sort those pieces from most important to least important.
Once ranked, the system splits the matrix into two groups:
- A significant matrix containing the top-contributing components, which gets quantized (compressed) lightly or not at all
- A less significant matrix containing lower-contributing components, which gets compressed aggressively to save memory and computation
Quantization is the process of representing numbers with fewer decimal places. Full-precision AI weights might use 32 bits per number; a quantized version might use 4 or 8 bits. Less data means faster processing and lower memory use, but it also means more approximation. The risk is that too much compression degrades the model's answers.
By applying different compression levels to different parts of the same layer, based on SVD's ranking, the patent aims to keep degradation concentrated in the parts of the network that can absorb it without affecting output quality.
The significant components undergo minimal or no quantization, while the less significant components undergo higher degrees of quantization.
Translation: Important parts stay high quality while unimportant parts are heavily shrunk down.
What this means for AI running on phones and laptops
For most people, this is about whether a capable AI model can run locally on your device rather than requiring a server somewhere else. Local AI is faster, more private, and works without an internet connection. The barrier has always been that the most capable models are simply too large for consumer hardware to run well.
If Microsoft can make this technique work reliably across a range of model sizes and hardware types, it becomes a practical tool for shipping AI features in Windows or other software without depending on cloud infrastructure. Microsoft's track record in AI efficiency patents suggests this is part of a broader push to make large language models viable on ordinary machines, not just in data centers.
Microsoft's 11th filing we've tracked since August in our on-device AI privacy watchlist follows earlier applications like scrubbing medical audio and battery-safe camera streaming.
Source: US 2026/0300700 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 8 sheets
8 drawing sheets from US 2026/0300700 A1 · click any drawing to enlarge

No comments yet