Nvidia Patents a Random-Bit Trick to Keep AI Training Math Accurate
When AI models train on low-precision math to save power and time, tiny rounding errors pile up and degrade accuracy. Nvidia's new patent proposes injecting controlled randomness into the scaling step to fight that drift.
What Nvidia's randomized number-scaling actually does
Imagine you're converting a high-resolution photo to a smaller file format. Every time you compress it, you lose a little detail, and if you do it repeatedly, the image gets noticeably worse. Training a large AI model has a similar problem: to run fast and use less memory, chips crunch numbers at lower precision, which introduces tiny rounding errors that build up over millions of calculations.
Nvidia's patent describes a way to fight this by adding a dash of randomness to how numbers inside the model get scaled up or down. Instead of always rounding the same direction, the system uses randomly generated bits to vary how scaling happens across different chunks of the calculation, which spreads the rounding errors around rather than letting them accumulate in one place.
The result is that you get the speed and efficiency benefits of low-precision math without paying as steep a price in model accuracy. That tradeoff sits at the center of every big AI training run today.
How random bits adjust scaling factors across tensor blocks
Neural networks store their internal values as tensors (essentially large grids of numbers). During training, those numbers are multiplied, added, and adjusted billions of times. To keep this manageable, modern AI chips use low-precision formats like FP8 or INT8, which represent numbers with fewer bits than standard floating-point math, trading some accuracy for a big jump in speed and memory efficiency.
The catch is quantization error: when you squeeze a high-precision number into fewer bits, you have to round it, and rounding always introduces a small inaccuracy. When the same rounding bias repeats across every calculation, those errors add up rather than cancel out.
Nvidia's patent addresses this through randomized block-level scaling. The processor divides tensors into blocks (portions), assigns each block a scaling factor that controls how its values are represented, and then perturbs those scaling factors using randomly generated bits. This is a hardware-level version of a technique called stochastic rounding, where randomness makes rounding errors statistically independent so they average toward zero instead of stacking in one direction.
The key claims point to circuits inside the processor itself handling the randomization, which means it happens at the chip level without extra software overhead.
What this means for training large AI models on less hardware
Nvidia's entire competitive position in AI infrastructure rests on squeezing more training performance out of each chip. Techniques that let you use lower-precision math without hurting model quality are worth real money: they mean faster training runs, lower energy costs, and smaller GPU clusters for the same result. If this approach works as described, it would let data centers train bigger models on the same hardware or cut costs on current-scale models.
For you as an AI user, the downstream effect would be models that are trained more efficiently, potentially making the economics of running large AI services a little less eye-watering. This also fits squarely into Nvidia's ongoing push with FP8 and microscaling formats in its Hopper and Blackwell chip families.
This is a focused, narrowly scoped hardware patent that addresses a real and well-known problem in AI training. It's not a flashy product announcement, but randomized scaling at the circuit level is a legitimate engineering target, and the four inventors include Paulius Micikevicius, who is one of the key architects behind mixed-precision training. That pedigree makes this worth taking seriously.
The drawings
29 drawing sheets from US 2026/0219841 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.