IBM Patents a System That Automatically Balances AI Processing Power to Cut Energy Costs
Running large AI models on a data center full of GPUs is expensive and wasteful if those GPUs are either sitting idle or overloaded. IBM's latest patent describes a feedback loop that continuously right-sizes how GPU computing power is carved up, based on live traffic and energy use.
How IBM's GPU balancing act keeps AI servers efficient
Imagine a restaurant kitchen where you can either have one chef cooking a big banquet (all the cooking power focused on one job) or split the kitchen into several smaller stations, each handling a different table's order at the same time. When the restaurant is quiet, you want the big-banquet setup. When it's slammed with orders, you want the multi-station approach. IBM's patent automates exactly that kind of decision for the computers that run AI chatbots and similar tools.
Modern AI chips called GPUs can be physically divided into smaller virtual slices (called MIG partitions) so multiple requests can be handled in parallel, or kept whole for maximum speed on a single task. The system IBM describes watches three things in real time: how many AI requests are coming in, how many words of output the system is generating per second, and how much electricity is being consumed. It then calculates whether the current GPU setup is the right one, and switches configurations automatically if it isn't.
The goal is to hit a target service level (requests per second, output speed) without burning more electricity than a set budget allows. No human needs to manually intervene every time traffic spikes or dips.
How the scaling factors trigger GPU partition switches
The patent describes an iterative control loop running on a computer system that manages a pool of GPUs handling LLM inference (the process of generating AI outputs from a prompt, as opposed to training the model in the first place).
Each cycle of the loop computes five ratios:
- Traffic Factor (TF): current incoming request rate divided by the target rate. A TF above 1 means demand is outpacing the goal.
- Throughput Factor (TPF): target token output divided by current token output. A TPF above 1 means the system is generating text more slowly than desired.
- Power Factor (PF): power budget divided by current consumption. A PF below 1 means the system is already over budget on electricity.
- GPU Scaling Factor (GSF) and MIG Scaling Factor (MSF): derived numbers that tell the system how aggressively to scale whole GPUs versus GPU partitions.
Based on these ratios, the system decides whether to convert a MIG (Multi-Instance GPU) service, where one GPU chip is split into smaller slices, into a full-GPU service (dedicating an entire chip to one model instance), or vice versa. This is the core switching action the patent protects.
The formulas ensure that the switching decision accounts for all three constraints simultaneously: throughput, traffic volume, and power draw, rather than optimizing for just one at the expense of the others.
What this means for the cost of running large AI models
The cost of running AI inference at scale is one of the biggest operational headaches for any company deploying large language models. GPUs are expensive to buy and expensive to power, and most deployments either over-provision (wasting money) or under-provision (degrading your experience as a user). A closed-loop system that adjusts GPU partitioning in real time could meaningfully reduce both waste and service degradation without human operators watching dashboards around the clock.
For IBM, this fits squarely into its cloud and enterprise AI infrastructure business. The power budget constraint is particularly notable: data centers are increasingly bumping against electricity limits, and a system that treats energy as a first-class input alongside performance targets reflects where the industry is heading.
This is infrastructure plumbing, not a headline AI feature, but it's the kind of plumbing that actually determines whether running AI at scale is economically viable. The three-way balancing act between traffic, throughput, and power budget is more sophisticated than simple auto-scaling, and the MIG partitioning angle is specific enough that this is unlikely to be obvious prior art. Worth watching if you follow cloud AI cost efficiency.
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.