Nvidia Patents a Random-Bit Trick to Keep AI Training Math Accurate
When AI models train on low-precision math to save power and time, tiny rounding errors pile up and degrade accuracy. Nvidia's new patent proposes injecting controlled randomness into the scaling step to fight that drift.
What Nvidia's randomized number-scaling actually does
Imagine you're converting a high-resolution photo to a smaller file format. Every time you compress it, you lose a little detail, and if you do it repeatedly, the image gets noticeably worse. Training a large AI model has a similar problem: to run fast and use less memory, chips crunch numbers at lower precision, which introduces tiny rounding errors that build up over millions of calculations.
Nvidia's patent describes a way to fight this by adding a dash of randomness to how numbers inside the model get scaled up or down. Instead of always rounding the same direction, the system uses randomly generated bits to vary how scaling happens across different chunks of the calculation, which spreads the rounding errors around rather than letting them accumulate in one place.
The result is that you get the speed and efficiency benefits of low-precision math without paying as steep a price in model accuracy. That tradeoff sits at the center of every big AI training run today.
How random bits adjust scaling factors across tensor blocks
Neural networks store their internal values as tensors (essentially large grids of numbers). During training, those numbers are multiplied, added, and adjusted billions of times. To keep this manageable, modern AI chips use low-precision formats like FP8 or INT8, which represent numbers with fewer bits than standard floating-point math, trading some accuracy for a big jump in speed and memory efficiency.
The catch is quantization error: when you squeeze a high-precision number into fewer bits, you have to round it, and rounding always introduces a small inaccuracy. When the same rounding bias repeats across every calculation, those errors add up rather than cancel out.
Nvidia's patent addresses this through randomized block-level scaling. The processor divides tensors into blocks (portions), assigns each block a scaling factor that controls how its values are represented, and then perturbs those scaling factors using randomly generated bits. This is a hardware-level version of a technique called stochastic rounding, where randomness makes rounding errors statistically independent so they average toward zero instead of stacking in one direction.
The key claims point to circuits inside the processor itself handling the randomization, which means it happens at the chip level without extra software overhead.
What this means for training large AI models on less hardware
Nvidia's entire competitive position in AI infrastructure rests on squeezing more training performance out of each chip. Techniques that let you use lower-precision math without hurting model quality are worth real money: they mean faster training runs, lower energy costs, and smaller GPU clusters for the same result. If this approach works as described, it would let data centers train bigger models on the same hardware or cut costs on current-scale models.
For you as an AI user, the downstream effect would be models that are trained more efficiently, potentially making the economics of running large AI services a little less eye-watering. This also fits squarely into Nvidia's ongoing push with FP8 and microscaling formats in its Hopper and Blackwell chip families.
This is a focused, narrowly scoped hardware patent that addresses a real and well-known problem in AI training. It's not a flashy product announcement, but randomized scaling at the circuit level is a legitimate engineering target, and the four inventors include Paulius Micikevicius, who is one of the key architects behind mixed-precision training. That pedigree makes this worth taking seriously.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
29 drawing sheets from US 2026/0219841 A1 · click any drawing to enlarge
Show the other 21 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →