OpenAI Patents AI Hardware That Skips Calculations It Knows Will Be Zero
A huge fraction of the math inside an AI chip is pointless: multiply anything by zero and you already know the answer. OpenAI's new patent describes hardware that detects those zero-valued inputs before the multiplication even starts, then simply skips the work entirely.
What OpenAI's zero-skipping chip design actually does
AI chips spend enormous amounts of energy doing multiplication, and a surprising share of those multiplications involve a zero, which means the answer is always zero before the first calculation begins. Today's chips mostly do that math anyway, burning power on a result that was never in doubt.
OpenAI's patent describes a chip design that watches for zeros as numbers flow in. When it spots one, special circuitry flags the upcoming multiplication as pointless and routes around it, sending a pre-determined zero straight to the output. It can even shut off power to the parts of the chip that would have done that math.
The design is tuned for how AI models actually work: the model's learned weights (the numbers baked in during training) stay loaded in place while a stream of fresh input values flows through. Because the weights don't change, the chip can decide once which multipliers to power down for a given weight, then hold that state for every input that follows.
… zero detection circuitry configured to generate: zero-weight indicators based on the plurality of weight values, and zero-activation indicators based on the plurality of activation values …
Translation: Special circuits check the data in advance to spot which numbers are set to zero.
How the bypass circuitry detects and skips zero multiplications
The patent describes a digital compute-in-memory (DCIM) architecture, meaning the math happens close to where the numbers are stored rather than shuttling data back and forth across a chip, which wastes both time and energy.
At the center of the design is a set of dedicated detection circuits:
- Zero-weight indicators: flags set when a stored weight value equals zero
- Zero-activation indicators: flags set when an incoming input value equals zero
- Selector circuitry: logic that checks both flags and, if either is zero, writes a zero to the output without touching the multiplier
- Power gating circuitry: hardware switches that cut power to multipliers and adder circuits when they are not needed
The architecture uses a weight-stationary dataflow (a pattern where the model's learned weights sit still while inputs stream past them). Because the weights don't move, the chip can evaluate the zero-weight flags once per weight-load event and keep the affected multipliers powered off for the entire duration of that batch, which is a more efficient use of power gating than checking every cycle.
When both a weight and an activation are non-zero, the multipliers fire normally. When either is zero, the selector bypasses the multiplier chain entirely, and the power-gating hardware keeps the idle circuits dark.
Power gating circuitry selectively disables power to multipliers and adders based on zero-weight indicators.
Translation: The hardware shuts off electricity to parts of the chip that would otherwise waste energy calculating zeros.
What this means for the cost of running AI on custom silicon
Power consumption is one of the central problems in AI hardware right now. Training and running large models consumes enormous amounts of electricity, and a chip that can skip a meaningful fraction of its own work without losing accuracy is genuinely useful at scale. The patent targets a property that is common in AI workloads: many weight values in trained models are zero or become zero after quantization, and sparse or zero-heavy activations appear frequently too.
For OpenAI, which operates large amounts of inference infrastructure, a chip design that reduces wasted computation per token could have a real effect on operating costs. This particular filing sits squarely in the custom-silicon space, where newest Big Tech patents on AI chip efficiency are rapidly accumulating as companies try to reduce their dependence on third-party hardware.
OpenAI's fourth chip patent we've tracked in our AI chip wars watchlist since July follows one on precise calculations and one on auto-switching math.
Getting from a legal idea filing to a working custom processor is a long road. You still need physical design, a factory partner, enormous production costs, and real hardware testing before any of this circuitry does anything outside a computer simulation.
The core concept, skipping math operations that multiply by zero, is already well known in academic research on lean artificial intelligence systems. What is new here is the specific way the inventors combined two particular power-saving tricks, not the underlying idea itself.
That said, one of those tricks maps directly onto how modern AI models actually run in the real world, which suggests the inventors were solving a practical problem, not a made-up one.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0244403 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →