Samsung Files Patent for Giving AI Services Only as Much Computing Power as They Need
Samsung's patent trains one AI to predict how slow another AI will be, then uses that prediction to decide how many copies of the second one to run.
What Samsung's AI server-sizing patent does
Every time you ask a chatbot to summarize a document, a rack of expensive chips somewhere has to do the work, and the company running it has to guess how many chips to keep switched on. Too few and your answer crawls in. Too many and the company pays for hardware that sits idle.
Samsung's patent describes a way to stop guessing. A service might chain several AI programs together (one that reads a PDF, another that writes the answer, say). The system builds a small forecasting model that predicts how long answers will take, given how many people are asking and how many copies of each program are running.
Then it searches for the smallest number of copies that still keeps answers under a time limit a manager sets. When traffic rises or falls, it can run the math again.
… identifying at least one optimal number of instances for each of the at least one AI model by performing an optimization procedure for the loss function using the number of the user requests for the first service …
Translation: The system figures out the exact computing resources needed based on how many people are using the service.
How the response-time model picks a server count
The patent covers a method for deciding how many instances (running copies) of each AI model a service should keep alive. Each instance takes up chips, usually GPUs (graphics processors that handle AI work), so more copies mean faster answers but a bigger bill.
Step one is a response time model, a separate AI that takes two inputs, the number of user requests and the number of copies of each model, and outputs how long a response will take. The filing describes training it by running test traffic against the service, adding copies until the speed gains become small, then repeating with more requests and different request types.
Step two is a loss function (a score where lower is better). It combines two things:
- The total number of copies across all the models, which rises as you add more.
- A penalty that switches on only when the predicted response time is longer than the allowed time.
Step three uses gradient descent (a method that nudges the numbers step by step toward a lower score) until the score stops changing much. The result is the number of copies for each model. The system can launch them itself or ask another machine to do it, and the filing mentions Kubernetes (software that manages server programs). It can rerun periodically or when requests rise or fall. The allowed time can be a target such as 95% of responses arriving within about 100 milliseconds, and a manager can relax that target instead of adding chips.
Why AI answers could slow down less at rush hour
The practical payoff is for you at busy moments. If a service can predict a slowdown and add capacity before it hits, your chatbot or document tool is less likely to stall when everyone shows up at once.
For the company, the same math cuts waste. AI chips are expensive to buy or rent, and the filing's goal is fast responses with relatively few resources. It also gives operators a clear choice when traffic jumps: add chips to hold the speed, or accept a slightly slower target and keep the hardware bill the same. The filing is written for cloud servers running AI services and does not tie the idea to any specific Samsung product.
Samsung's 64th filing we've tracked since June in our AI chip wars watchlist follows one cutting data-shuffling time and memory chips doing AI math.
You will probably never see this patent at work, and that is the point. The best result is a chatbot that answers at the same speed at 9 a.m. on a Monday as it does at 3 a.m. on a Sunday.
The failure it targets is familiar: a spinner that hangs when everyone piles in at once. Keeping every chip running around the clock avoids that, but someone pays for the idle hardware, and eventually that shows up in subscription prices. A system that predicts slowdowns and adds capacity just ahead of them keeps the speed and trims the bill.
The filing is routine cloud plumbing with no phone or Galaxy feature attached. It is sensible, useful engineering that you only notice when it fails.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
16 drawing sheets from US 2026/0350397 A1 · click any drawing to enlarge
Show the other 8 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in