A New Microsoft Patent Tests Which Parts of an AI Can Run on Cheaper Math
Microsoft's patent tests an AI model one piece at a time to find which parts can get by on rougher math. The goal is a model that is cheaper to run and still gives the same quality of answers.
What Microsoft's AI math-trimming patent does
You're packing a suitcase that's too heavy, so you start pulling things out. The trick is knowing what you can leave behind without ruining the trip. Keep the toothbrush, ditch the fourth pair of shoes.
AI chatbots have the same problem. Behind the scenes they do a staggering amount of arithmetic, and every number is stored with some level of detail, a bit like how many decimal places you keep. More detail costs more memory and more computing power. Cutting the detail everywhere makes the answers worse.
This Microsoft patent application describes a testing tool that figures out where to cut. It lowers the detail on one piece of the AI at a time, checks how much the answers suffer, and ranks the pieces from most sensitive to least sensitive. Then it recommends full detail where it matters and less detail everywhere else, so the AI runs cheaper without getting noticeably worse.
a test configuration identifier stored in memory and executable by a processing system to identify a plurality of mixed-precision test configurations that assign mixed-precision data types among components of the language model …
Translation: The system finds different ways to mix high and low quality math across parts of the AI.
How Microsoft ranks which AI parts matter
Large language models (the AI behind chatbots) are built from stacks of layers called decoders, and most of their work is matrix multiplication (huge grids of numbers multiplied together). Normally every number is stored in the same data type, meaning the same number of bits, such as FP8 or FP4. Newer AI chips support lower-bit types that are faster and lighter, but using them everywhere hurts accuracy.
The patent describes three software pieces:
- Test configuration identifier: reads the model's layout and the chip's list of supported data types, then builds test setups that each give one component a lower-bit type while the rest stay higher.
- Salience analyzer: runs the model under each setup, either in an emulator (software that mimics the chip) or on the chip itself, and measures a saliency metric, meaning how much that one change disturbs the model. It can measure noise added to the math, or the drop in accuracy on test tasks like Q&A, coding or math, compared with the original setup. It then ranks the components by sensitivity.
- Quantized model recommender: gives higher-bit types to the most sensitive components and lower-bit types to the least sensitive, and can then set up the chip to match.
The filing also describes a two-round version. The first round ranks whole decoders. The second ranks individual channels (columns of a weight grid) inside the most sensitive decoders, so even those can be partly trimmed. It also describes using three tiers of precision instead of two.
The recommended mixed-precision data type configuration assigns a first, reduced-precision data type to a least salient subset of the components identified in the saliency ranking while assigning a second, higher-precision data type to a most salient subset of the components identified in the saliency ranking.
Translation: It puts the cheap math where it hurts least and keeps the precise math where it matters most.
Why cheaper-to-run AI models reach you
Running AI costs real money, mostly in chips, memory and electricity. A method that shows which parts of a model can safely use cheaper math could lower that bill. For you, that could eventually mean faster replies, lower prices, or AI features that fit on smaller machines. The filing keeps accuracy front and center, aiming to stay within a target threshold of the original model's quality, and sometimes with no accuracy loss at all.
The filing gives no numbers for how much it would save, and it describes a testing process rather than a product. Still, it goes after a problem every company running large AI models faces, and it turns the question of where to cut into a repeatable, ranked test. That makes it a useful window into how Microsoft thinks about AI running costs.
Microsoft's 21st filing we've tracked since July in the AI chip wars continues its work on AI efficiency, following one cutting memory costs and one routing by answer length.
You will never see this working. If it does its job, you would notice it as an AI service that answers a bit faster or costs a bit less, with no drop in answer quality.
The idea is easy to follow: lower the detail on one piece, see what breaks, and keep full detail only where it hurts. That plain logic is a point in its favor, because the person at the end of the chain gets the savings without accepting worse answers.
The filing offers no figures for the speedup or the savings, so how much you would feel is an open question. For now it looks like sensible behind-the-scenes engineering that matters most to whoever pays the electricity bill for running AI.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0319938 A1 · click any drawing to enlarge
Show the other 1 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in