Qualcomm
Qualcomm · AI / ML · Filed May 9, 2025 · USPTO published Nov 12, 2026 · This week's release
Qualcomm Files Patent for Shrinking AI Models So They Fit on Your Phone
Shrinking an AI model to run on a phone has a catch: a handful of oversized numbers can throw off the whole model. Qualcomm's new patent application describes a way to tame them before the model gets compressed.
- Publication number
- US 2026/0359717 A1(US20260359717A1)
- Applicant
- QUALCOMM Incorporated
- Filed · Published
- May 9, 2025 · Nov 12, 2026
- Inventors
- Boris VAN BREUGEL, Markus NAGEL, Yelysei BONDARENKO
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 706/12
What Qualcomm's AI-shrinking patent actually does
To run an AI chatbot on a phone, the model's numbers have to be rounded off into smaller, simpler ones. That saves memory and battery, but there is a catch: a few unusually huge values stretch the rounding scale so far that everything else gets blurred, and the model gets worse at its job.
Qualcomm's filing describes a fix. Before the rounding happens, the model's numbers get rotated and rescaled so the extreme values shrink toward the rest of the pack. A matching reversal is built in, so at full precision the model still gives the same answers.
Once that's done, the adjustments are folded into the model's existing numbers, so your device does no extra work. The goal is a model that is smaller and faster without the usual accuracy hit.
… combine the first weights and the second weights with corresponding weights in the trained machine learning model to generate combined weights; and generate a quantized machine learning model based on the combined weights.
Translation: It merges old model weights with new transformation data to create a smaller, compressed model.
How the outlier numbers get flattened out
The filing covers two related ideas for quantization (storing a model's numbers in a smaller format, such as 8-bit whole numbers instead of 16-bit or 32-bit decimals). Both go after outliers, values far bigger than the rest, which force a coarse rounding scale on everything else.
The first idea adds a matched pair of transformations to a trained model: one on an input, and its inverse on the output. In the attention blocks (the part of a language model that decides which words matter most), the filing pairs key weights with query weights, or value weights with an inverted copy of themselves. The transformations are built from small blocks that scale and rotate values within a subset of channels (slices of the data). Their settings start as do-nothing values and are trained to cut rounding error, then merge into the model's existing weights.
The second idea targets the residual stream (the path that carries data forward past each layer):
- Compute a scalar (a single scaling number) from the normalization layer.
- Normalize the residual, then multiply it value by value with the activations (the numbers passing between layers).
- Hand the scalar to the next layer so the scaling stays consistent through the model.
The filing also describes fine-tuning the shrunken model against the full-precision one, in a teacher and student setup.
… perform an elementwise multiplication on the normalized residual and the activations based on a scalar of the normalization layer to generate rescaled activations; and quantize weights of the machine learning model based on the rescaled activations …
Translation: It scales down internal data values so the entire model can be simplified without losing accuracy.
What this means for AI that runs on your phone
If you've wanted an AI assistant that works on your phone without sending your questions to a server, shrinking models is the hard part. Smaller numbers mean less memory, less power and faster answers. The filing says a model treated this way can be quantized to 4 or 8-bit precision without degrading performance, and it points to privacy as a perk of keeping small models on the device.
The filing also notes that mixing different precisions layer by layer can force extra conversions that add delay and power use. This approach goes after the outliers directly. It's a patent application, and no product is named, so what you'd see on a device depends on choices outside this document.
Qualcomm's 65th filing we've tracked in our AI chip competition since July adds to a run that includes one cutting data shuffling in graphics chips and one easing AI load on Wi-Fi routers.
Source: US 2026/0359717 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Qualcomm patent news breaks.
Add Patentlyze on Google
One click.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 13 sheets
13 drawing sheets from US 2026/0359717 A1 · click any drawing to enlarge
Show the other 5 sheets
More from Qualcomm
All 624 Qualcomm patents →
More in AI training & infrastructure
All 316 AI training & infrastructure patents →
Microsoft
A New Microsoft Patent Tests Which Parts of an AI Can Run on Cheaper Math
Microsoft
Microsoft Patents a Way to Run Giant AI Models on Chips Too Small to Hold Them
Nvidia
Nvidia Patents a Method to Run AI Processing Steps on Separate Chips
IBM
No comments yet