Get the newsletter
Microsoft · AI / ML · Filed Mar 31, 2025 · USPTO published Oct 1, 2026 · Oct 1 release

Microsoft Patents a Way to Compress AI Models Without Losing What Matters Most

Running a powerful AI model on a phone or laptop usually means accepting a watered-down version. Microsoft's new patent tries to change that math by being surgical about what gets compressed and what doesn't.

A system for compressing AI models connects user devices, data sources, and servers through a network. Drawing from patent filing US 2026/0300700 A1.
A system for compressing AI models connects user devices, data sources, and servers through a network.All 8 drawings ↓
Publication number
US 2026/0300700 A1(US20260300700A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Mar 31, 2025 · Oct 1, 2026
Inventors
Yehonathan REFAEL KALIM, Adam HAKIM, Gilad KIRSHENBOIM
Status when we published
Waiting for an examiner
Document
20 claims
US classification
706/25

How Microsoft shrinks AI to fit on everyday hardware

You're trying to run an AI assistant directly on your phone, no cloud required, but the model is simply too large for the hardware to handle at full quality. Something has to give.

Microsoft's patent describes a technique for deciding which parts of an AI model are worth preserving in high fidelity and which parts can be safely squeezed down without the model noticing. Think of it like digitizing an old photograph: you keep the subject's face sharp and let the background go a little blurry. The AI keeps its important "neurons" at full precision and compresses the rest aggressively.

The result is a model that takes up far less memory and computing power, making it practical to run on a laptop, phone, or other edge device that can't tap a data center. The goal is to preserve accuracy where it counts while making the tradeoffs in places the model can absorb.

From the filing · CLAIM 1
… identifying a first plurality of significant columns and/or rows in the orthogonal matrix that make above a threshold contribution to producing an accurate final inference …

Translation: It finds the parts of the AI that actually matter for making correct guesses.

How the patent separates critical AI weights from disposable ones

The patent centers on a mathematical technique called Singular Value Decomposition (SVD), a method for breaking a large table of numbers (called a matrix) into simpler pieces ranked by how much they contribute to the matrix's overall meaning. In an AI model, each layer of the network contains such matrices, and SVD lets you sort those pieces from most important to least important.

Once ranked, the system splits the matrix into two groups:

  • A significant matrix containing the top-contributing components, which gets quantized (compressed) lightly or not at all
  • A less significant matrix containing lower-contributing components, which gets compressed aggressively to save memory and computation

Quantization is the process of representing numbers with fewer decimal places. Full-precision AI weights might use 32 bits per number; a quantized version might use 4 or 8 bits. Less data means faster processing and lower memory use, but it also means more approximation. The risk is that too much compression degrades the model's answers.

By applying different compression levels to different parts of the same layer, based on SVD's ranking, the patent aims to keep degradation concentrated in the parts of the network that can absorb it without affecting output quality.

From the filing · THE ABSTRACT
The significant components undergo minimal or no quantization, while the less significant components undergo higher degrees of quantization.

Translation: Important parts stay high quality while unimportant parts are heavily shrunk down.

What this means for AI running on phones and laptops

For most people, this is about whether a capable AI model can run locally on your device rather than requiring a server somewhere else. Local AI is faster, more private, and works without an internet connection. The barrier has always been that the most capable models are simply too large for consumer hardware to run well.

If Microsoft can make this technique work reliably across a range of model sizes and hardware types, it becomes a practical tool for shipping AI features in Windows or other software without depending on cloud infrastructure. Microsoft's track record in AI efficiency patents suggests this is part of a broader push to make large language models viable on ordinary machines, not just in data centers.

Microsoft's 11th filing we've tracked since August in our on-device AI privacy watchlist follows earlier applications like scrubbing medical audio and battery-safe camera streaming.

Source: US 2026/0300700 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 8 sheets

8 drawing sheets from US 2026/0300700 A1 · click any drawing to enlarge

AI model compression system overview., drawing sheet 1 from US 2026/0300700 A1
Sheet 1 AI model compression system overview.
Training and quantization process., drawing sheet 2 from US 2026/0300700 A1
Sheet 2 Training and quantization process.
Quantization during model training., drawing sheet 3 from US 2026/0300700 A1
Sheet 3 Quantization during model training.
Matrix decomposition and quantization., drawing sheet 4 from US 2026/0300700 A1
Sheet 4 Matrix decomposition and quantization.
Quantizing neural network layers., drawing sheet 5 from US 2026/0300700 A1
Sheet 5 Quantizing neural network layers.
Differentially quantizing neural network layers., drawing sheet 6 from US 2026/0300700 A1
Sheet 6 Differentially quantizing neural network layers.
Processing input with quantized network., drawing sheet 7 from US 2026/0300700 A1
Sheet 7 Processing input with quantized network.
Computer system components., drawing sheet 8 from US 2026/0300700 A1
Sheet 8 Computer system components.
Patent filing page

More in AI training & infrastructure

All 316 AI training & infrastructure patents →

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.