Google's New Patent Trains AI by Cutting Out the Numbers That Don't Matter

By Patentlyze Team · Updated Jul 10, 2026

Most of the numbers inside a large AI model barely influence its answers. Google has filed a patent for a training technique that finds those useless numbers, zeros them out, and then trains the model again without them.

How Google trims fat from a trained AI model

Imagine you're a chef who has memorized 10,000 recipes, but you only ever cook at a small café that serves 20 dishes. Most of what you memorized is dead weight. AI models have the same problem: they're trained on enormous datasets and end up full of internal calculations that contribute almost nothing to their actual job.

Google's patent describes a two-phase approach to fix this. First, the AI gets fine-tuned on the specific task you want it to do. Then the system identifies which parts of the model are doing the least work and stamps them out with what it calls a sparsity mask, basically a map of zeros that tells the model to skip those calculations entirely. Finally, the model gets fine-tuned a second time with those zeros locked in.

The result is an AI that is structurally lighter. It does less arithmetic to reach the same (or nearly the same) answer, which means it can run faster and on less powerful hardware. For a company running AI at Google's scale, even small efficiency gains across millions of requests add up fast.

How the sparsity mask prunes a model in two training passes

The patent describes a method called structured sparsity induction, applied during the fine-tuning phase of a pre-trained model rather than during the expensive original training run.

The key data structure here is a tensor (think of it as a large table of numbers that represents the model's internal weights, the numerical dials that control how the AI responds). The system groups those numbers into blocks and, within each block, forces at least one value to zero. That's the sparsity mask: a template that says "this position is always zero, don't compute it."

The training sequence looks like this:

• Fine-tune the pre-trained model on a target task (first pass)
• Analyze the model's tensors and generate the sparsity mask, identifying which weights are expendable
• Apply the mask, locking selected weights to zero
• Fine-tune again with the mask in place (second pass), so the remaining weights adjust to compensate for what was removed

The block-level constraint on where zeros go is intentional. Modern AI chips (like Google's own TPUs) are built to exploit regular patterns of zeros in structured ways, so this approach is designed to translate directly into real hardware speed gains, not just theoretical savings.

What leaner AI models mean for cost and deployment

Running large AI models is expensive. Every query sent to a model like Gemini involves billions of floating-point multiplications. A model that can skip a significant fraction of those multiplications because they've been pre-zeroed out costs less per query to operate, and can potentially run on smaller, cheaper chips.

This matters beyond Google's own data centers. If the technique works well, it's the kind of thing that could make capable AI models practical to run on-device, on a phone or laptop, rather than requiring a round-trip to a server. Google has been pushing hard on on-device AI (Gemini Nano, for example), and methods that shrink model computation without gutting accuracy are directly useful for that goal.

Editorial commentary on a publicly published patent application. Not legal advice.