Nvidia Patents a Fix for Chip Traffic Jams That Leave Some Data Waiting Forever
Nvidia's filing describes a traffic rule for chip-to-chip links: keep some shared capacity in reserve, so a sender with tiny packets can't crowd out everyone else.
What Nvidia's shared-credit reserve actually does
Every time one computer chip sends data to another, a tiny traffic cop decides whose packet goes out next. Each kind of traffic gets an allowance of credits, which work like permission slips from the receiving end: no credits, no sending.
To avoid giving every kind of traffic its own big waiting area, designers added a pool of shared credits. Nvidia's filing says that creates a trap. When the shared pool runs low and refills in dribbles, a sender with small packets grabs every scrap while senders with bigger packets wait indefinitely. The patent's fix is a reserve: the shared pool only opens up when it holds more than a set minimum, tied to the size of the largest packet. Below that line, everyone falls back on their own dedicated credits.
A second option protects time-sensitive traffic. In the filing's example, 10 of 100 shared credits are set aside for it and 90 go to everything else.
… block one or more of the selected requests from being passed from the first-level arbitration logic to the second-level arbitration logic responsive to determining i) a lack of available shared credits in a shared credit pool, or ii) a lack of available dedicated credits assigned to the respective traffic type.
Translation: The system stops data packets when there are no credits left to send them.
How the credit check sits between two rounds
The patent covers the part of a chip's data link that decides which waiting packet goes out next. It uses two-level arbitration: level one picks a winner within each traffic type (reads, writes, completions), and level two picks the final winner across types.
Between the two rounds sits a credit check. Credits are permission slips from the receiving end, which has limited buffer space. Each traffic type gets dedicated credits, and all types can also draw on a shared pool, which saves chip area compared with a separate buffer per type. PCIe, a common connection standard, already uses shared credits.
The catch is starvation. In the filing's example, one client always asks for 2 credits while two others ask for 4. Once the shared pool refills in small amounts, the 2-credit client takes every refill and the others never get through. The fix: a request passes only if the shared pool is above a minimum (tied to the largest packet size) or it fits within that type's dedicated credits. Level two uses a Weighted Least Recently Used scheme (the sender who went longest ago, weighted, goes next).
An extra option splits shared credits by group: 10 of 100 for time-sensitive (isochronous) clients, 90 for the rest, with refills stopping once a group hits its share.
Why GPU and switch links could care about fairness
For you, the payoff would be invisible: fewer stalls when GPUs, network cards and switches shuffle huge amounts of data around, as they do in AI servers. A link that lets small, frequent requests crowd out big ones can leave parts of a system waiting, and this filing is a guard against that.
The filing says the idea works on PCIe, NVLink, Ethernet, InfiniBand and chip-to-chip links, and the claims name CPUs, GPUs, data processing units, network adapters and switches. It also says a single shared buffer can replace one buffer per traffic type to save chip area, and the reserve rule is what makes that trade safe. That is the strategic angle: cheaper chip space without letting some traffic get stuck.
Nvidia's 65th patent application we've tracked since July in our AI chip wars watchlist follows earlier filings on faster chatbot answers and running big models on less memory.
Claim 1 is wider than the headline idea. It describes a transmitter circuit with a checkpoint between two rounds of picking who sends next, and it holds back a request when either the shared credits or that sender's own credits run out. The claim never mentions the minimum reserve that the description is built around.
That reserve appears in the narrower claims, including the independent claim about a transmitter device, which spells out the exact test: pass if shared credits exceed a minimum, or if the packet fits within the sender's own allowance. So the broad claim covers the general shape, and the specific cure for starvation sits a layer down.
Either way, this is plumbing inside chips, and nobody will notice it working. Claim 23 names GPUs, network adapters and NVLink switches, so the rule is meant to travel across a lot of hardware.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
13 drawing sheets from US 2026/0361665 A1 · click any drawing to enlarge
Show the other 5 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in