Nvidia Patents a Number-Rounding Shortcut for AI Math That Eats Chip Power
Every time an AI model turns raw scores into probabilities, the chip has to compute an exponential function, and doing that cleanly and quickly is harder than it sounds. Nvidia has patented a shortcut that swaps the expensive math for a sequence of simpler integer operations.
What Nvidia's exponential shortcut actually does for AI
Imagine an AI model has just read your question and produced a list of possible answers, each with a raw score. Before it can tell you which answer is most likely, it has to run every one of those scores through a specific math operation called an exponential function, turning them into percentages that add up to 100. That step happens billions of times a day across data centers.
The problem is that doing this math precisely is expensive, in terms of both time and energy. Nvidia's patent describes a way to approximate the result using a much simpler sequence of steps: shifting bits around inside a number (the chip equivalent of moving a decimal point) rather than doing the full calculation. The answer isn't perfect to the last decimal place, but for AI inference (when a trained model is making predictions, not still learning) it's close enough to not matter.
The same trick can be run in reverse to handle logarithms, the mathematical opposite of exponentials, giving chip designers one compact solution for both operations.
… increment the intermediate representation to cause a mantissa of the incremented intermediate representation to correspond to the fixed point representation; shift a decimal point of the incremented intermediate representation to produce an exponential approximation of the value …
Translation: The chip adjusts the internal math representation and moves the decimal point to quickly estimate complex exponential values.
How the bit-shift trick replaces expensive exponential math
The patent targets a specific step inside neural network inference called the softmax function, which converts a model's raw output scores into probabilities. Softmax requires computing the exponential (e raised to the power of each score), and doing that in hardware quickly without burning too much power is a real engineering challenge.
Nvidia's approach works in four stages:
- Take a floating-point value (a number with a fractional part, like 3.14) and convert it to a fixed-point representation, a simpler format that includes a correction called a bias factor to keep the result accurate.
- Use that fixed-point number to build an intermediate representation with a specific number of bits, then increment it (add one) so that the mantissa, the significant digits of the number, lines up with the fixed-point value.
- Shift the decimal point of that incremented number. Because of how floating-point numbers are structured at the hardware level, this shift is the exponential approximation, no full math required.
- Feed the result directly into the model's inferencing operation in place of the precisely computed exponential.
The whole process runs in half-precision floating-point format (a compact 16-bit number representation popular in AI hardware because it's fast and memory-efficient). Running the steps backward yields an approximation for the logarithm, so one hardware unit handles both directions.
Approaches presented herein provide for the efficient approximation of exponential or logarithmic functions using values in half-precision, floating point format. In at least one embodiment, an activation function of a neural network can perform tasks such as to convert raw scores from the network into probabilities using the exponential of each output.
Translation: This method helps AI models turn raw data into probability scores by using a faster way to calculate complex math functions.
What this means for the chips running your AI queries
For anyone using an AI assistant, a recommendation engine, or a translation tool, this kind of optimization is invisible but consequential. Faster, cheaper exponential math means the chip can process more requests for the same power budget, or the same number of requests on smaller, lower-power hardware. That matters most at the edge, in laptops, phones, or embedded devices where every milliwatt counts.
Nvidia already dominates the AI training chip market, but inference (running finished models at scale) is where the volume and the competition are fiercest. Squeezing more efficiency out of operations as common as softmax is exactly the kind of work that compounds into a real performance edge, and it fits neatly into the latest Big Tech patents on AI chip arithmetic, where companies are racing to do more math with less silicon.
That makes this Nvidia's 34th filing we've tracked since July in the AI chip wars, adding to work like faster training by sharing workload and models that resize to fit chips.
The cost of this approach is precision, and it only holds in one specific window: after a model is trained and already running. At that stage, small rounding errors don't change which answer the model picks, so the shortcut is safe. Run the same approximation during training, where tiny errors compound across millions of learning steps, and the model could degrade badly.
The other cost is narrowness. This solution is tuned for one specific 16-bit number format, so if the industry moves toward even smaller formats, the engineering work has to start over for each one. That is a recurring bill, not a settled problem.
The trade still reads as worth it. When the same calculation runs billions of times a day across thousands of servers, even a tiny saving per operation becomes a meaningful saving overall, and the precision being given up is precision that was not doing useful work anyway.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
18 drawing sheets from US 2026/0252664 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →