Nvidia Patents a Chip Design That Runs AI's Hardest Math Simultaneously Faster
Neural networks run on a lot of math that current chip circuits handle one piece at a time. Nvidia's new patent describes a way to split that work across multiple units running simultaneously, potentially cutting the time AI models spend on their most expensive calculations.
What Nvidia's parallel math circuit actually does
Think of your phone's calculator app. It handles one equation at a time. Now imagine if, instead of waiting for each answer before starting the next problem, your phone could solve dozens of equations all at once. That's roughly the idea here.
Neural networks, the engines behind AI tools like image generators and voice assistants, rely on a set of complex math functions (things like exponentials and logarithms) that are expensive for chips to compute. Nvidia's patent describes a chip design with multiple dedicated math units working in parallel, each tackling a slice of the problem at the same time instead of queuing up in sequence.
There's also a number-formatting trick involved. The circuit converts numbers into a compact format before doing the hard math, then converts them back to a usable format at the end. That keeps the calculations fast without losing too much accuracy, which is a constant balancing act in AI chip design.
How the SFUs convert formats and split the workload
The patent covers a processor design built around a cluster of Special Function Units (SFUs), which are dedicated circuits for computing non-linear math functions. These are the kinds of operations (sigmoid, softmax, GELU, and similar activation functions) that neural networks use constantly to decide how information flows through each layer.
The key design choice is parallelism. Instead of one SFU working through a long list of values sequentially, multiple SFUs divide the input vector (a list of numbers representing data) into chunks and process each chunk at the same time. The patent describes the circuit as operating "in parallel" on those portions simultaneously.
Before each chunk goes into an SFU, a quantizer (a circuit that converts numbers from one format to another) converts values from a general floating-point representation into a fixed-point format (a compact integer-style representation that is cheaper to compute). The SFU does its work in fixed-point, and a second conversion step translates the output back to the format the rest of the chip expects.
The result is a pipeline that trades some numerical precision for speed and power efficiency, a tradeoff that is standard practice in AI inference hardware where throughput matters more than perfect arithmetic.
What this means for AI chip performance
For Nvidia, this kind of low-level chip work is where GPU performance is actually won or lost. Non-linear functions are a bottleneck in neural network inference because they can't easily be broken into simple multiply-accumulate operations the way most of a network's math can. Dedicated, parallel hardware for these functions means less waiting, which translates directly into faster AI output and lower power draw per calculation.
If this design ships in a future GPU generation, you'd feel it as faster responses from AI-powered apps or more AI features fitting onto lower-power devices. It also fits Nvidia's broader push to squeeze more AI performance out of each watt of power, a priority as data center energy costs become a real constraint on scaling AI systems.
This is unsexy but important chip plumbing. Special function units have been part of GPU architecture for decades, but optimizing them specifically for neural network activation functions is genuinely useful work. It won't make headlines on its own, but this is the kind of incremental hardware improvement that compounds into meaningful performance gains across a GPU generation.
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.