A New Qualcomm Patent Has Your Phone Decide Which AI Questions Need the Cloud
Your phone could decide, question by question, whether its own small AI is good enough or whether the answer needs a bigger model in the cloud. Qualcomm's patent application describes a small router that makes that call before anything is answered.
What Qualcomm's on-phone AI router actually does
Today, an AI chatbot writes its answer one word at a time, and every single word takes a full trip through a huge model. That is why the biggest models live in data centers, while your phone carries a smaller one that can handle fewer kinds of questions. Send everything to the cloud and you wait on the round trip. Keep everything on the phone and you get more wrong answers.
Qualcomm's patent adds a small router, a traffic cop that runs on your phone and reads each question first. If the on-device model can probably handle it, the phone answers by itself. If not, the question goes to a bigger model elsewhere. In one example in the filing, a stock-price lookup stays on the phone.
The goal is to use the small model whenever it is good enough, and the big one only when it earns the extra cost.
… select, based on the input prompt and using a trained router machine learning model configured to execute on the edge device, whether to process the input prompt using a first machine learning model or a second machine learning model …
Translation: The phone uses a smart router model to decide whether to handle a prompt locally or send it elsewhere.
How the router grades a question before answering
The core claim covers a router, itself a trained machine learning model, that runs on the edge device (a phone, tablet or laptop). For each prompt, it chooses between a smaller model on the device and a larger one, typically in the cloud. The device then answers locally or forwards the prompt.
To train the router, the filing runs test questions through both models and labels each one by which model got it right. A question the small model flubbed gets labeled “cloud.” At work time, the router squeezes the prompt into one summary using pooling (combining per-word numbers into a single set) and outputs a confidence score: its guess that the small model will get it right. Below a threshold, the prompt goes to the cloud.
The filing also describes ranking prompts by complexity, based on how many tool calls (requests to other apps or services) an answer needs, and choosing the smallest model that clears a quality bar.
In one variation, the router shares the first layers of the small model and adds low rank adaptation (a lightweight tweak) to save compute. In the filing's tests, sending about 25 to 30 percent of questions to the cloud recovered roughly 85 percent of the quality gap.
The second machine learning model has a larger size than the first machine learning model.
Translation: The second model is much bigger than the first one.
Why your phone's AI could get faster and cheaper
If this works the way the filing describes, you would mostly notice speed. Simple questions, like a stock price or a quick fact, could be answered on your phone without waiting on a trip to a data center, while only the hard ones pay that delay. The filing also points to lower energy use and less strain on the phone's memory and processors.
For Qualcomm, which makes the phone chips the filing describes (a processor with a built-in AI unit), the idea fits a phone that does more of the work itself. It matters to whoever runs the big model, too: every question handled on the phone is one the data center never has to process.
This is the tenth Qualcomm filing we've tracked in our AI models working in teams since July, adding to work like the text draft control one and the photo model picker.
Of everything in this filing, the router is the part closest to a shippable feature, because it is software. It needs no new chip: the description lists an ordinary phone processor with a built-in AI unit as the place it runs. What has to exist first is a small model on the phone and a bigger one in the cloud, and that pairing is already a familiar setup.
The slow part is the homework. The router learns from test questions that were run through both models and graded, and the filing's tests mostly check whether a model picked the right tool and the right details. Grading open-ended chat answers is harder, and the filing does not show that case.
Verdict: a practical idea that looks close to ready. Its payoff depends on how little the router itself costs to run, which is exactly what the shared-layers version tries to shrink.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Qualcomm patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0340513 A1 · click any drawing to enlarge
Show the other 4 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in