Microsoft Patents a Pre-Calculated Answer Sheet to Speed Up AI Math

By Patentlyze Team · Updated Jul 21, 2026

The most expensive thing a modern AI chip does, billions of times per second, is multiply giant tables of numbers together. Microsoft's new patent describes a way to pre-compute chunks of that work so the chip can look up answers instead of recalculating them.

What Microsoft's matrix multiplication shortcut actually does

Every AI model, whether it's a chatbot or an image generator, runs on a single operation repeated at enormous scale: multiplying enormous grids of numbers (called matrices) together. The faster a chip can do that, the faster and cheaper the AI runs.

Microsoft's patent describes a shortcut. Instead of doing all the multiplication fresh every time, the system pre-calculates a big set of possible partial answers and stores them in a lookup table, basically a cheat sheet. When it needs a result, it just finds the right row in the cheat sheet and grabs the answer.

The twist here is that the system mixes two different levels of numeric precision, using low-precision "weight" numbers (the stored knowledge of the model) alongside higher-precision "activation" numbers (the live data being processed). The lookup table bridges the gap between those two different sizes, letting the chip stay fast without losing accuracy.

How the lookup table pre-computes and retrieves partial results

The patent targets mixed-precision matrix multiplication, a technique where different parts of an AI model's calculation use different numeric resolutions to save memory and compute. The challenge is that standard hardware isn't always efficient at mixing those resolutions on the fly.

Here's the specific approach:

• A low-precision weight matrix (the AI model's learned parameters, stored in n bits) is broken apart into n separate one-bit matrices, stripping it down to its individual binary layers.
• A higher-precision activation matrix (the data currently flowing through the model, in m bits where m is larger than n) is grouped into small chunks.
• The system pre-generates a lookup table covering every meaningful combination of those activation chunks with the one-bit weight slices. This is the cheat sheet: partial answers computed once, stored for reuse.
• When a new weight matrix arrives, the method uses chunks of it as indices (essentially addresses) to find the right pre-computed answers in the table, then adds those answers together into a final result.

The net effect is that expensive multiplication operations get replaced by cheaper memory lookups and additions, which are much faster on modern silicon.

What this means for AI chips and inference costs

AI inference, the step where a trained model actually answers your question, is dominated by the cost of matrix math. Any method that reduces how much of that math runs on actual multiply circuits, and replaces it with table lookups and additions, can meaningfully cut latency and power draw on AI accelerators.

This is especially relevant as the industry pushes toward running AI models locally on devices (phones, laptops, edge servers) where power and chip area are tightly constrained. A lookup-table approach that efficiently handles mixed-precision arithmetic could make it easier to run larger models on smaller hardware without the quality loss that comes from simply rounding everything down to low precision.

Editorial commentary on a publicly published patent application. Not legal advice.