Qualcomm Patents a Shortcut to Make Phone Chatbots Answer Sooner
Before a phone chatbot says a single word back to you, it has to read your whole question. Qualcomm's filing splits that reading into its own lighter pass and saves the costly word-picking step for the end.
What Qualcomm's two-part chatbot split does
You're typing a long question into a chatbot on your phone, and then you wait. The phone has to read every word of what you wrote before it can start replying, and on a small device that reading takes a while.
Qualcomm's filing describes a way to trim that wait. It splits the chatbot's model into two parts. The first part reads your question one small piece at a time and jots down notes about each piece. The second part, which does the actual word-picking for the reply, only switches on once the reading is done. The reading stage skips the word-picking machinery entirely.
Those notes are kept, so if you ask a follow-up question, the phone can pick up where it left off instead of re-reading your whole conversation.
… determine, by a first language model, a first internal state for each of the first set of input tokens; and process, by a second language model, responsive to the first internal state being determined for all of the first set of input tokens, the first set of input tokens and the first internal state for each of the first set of input tokens …
Translation: One AI model processes the input text first, and then hands its internal data directly to a second model to speed things up.
How the prompt gets read before any word is picked
Chatbots read your text as tokens, small chunks that average about four characters. As the model works through them, it keeps a running record called the internal state, or KV cache (a stored set of notes on what it has already read).
The filing splits the model in two. Part 1, the decoder layers (the core number-crunching steps), loops through your tokens one at a time and builds the notes for each. It skips the LM head, the final stage that turns the model's internal numbers into an actual word.
When the last token is in, the tokens and notes go to Part 2, which includes the LM head and generates the reply. For a follow-up message, the loop runs on the new text, and the earlier notes and reply tokens are joined with the new ones before Part 2 runs again.
The description contrasts this with padding every prompt to a fixed length, which took an example 2.2 seconds regardless of your question's size, partly because some phone neural processors only accept fixed-size inputs. The token-at-a-time loop scales with your prompt, at an example 100 milliseconds per token.
A processor-implemented method of accelerating prompt inferencing of large language models includes receiving a first input prompt.
Translation: The patent describes a new way to make large artificial intelligence models respond to users much faster.
Why on-phone AI replies could start sooner
If you've used an AI assistant on a phone, you know the wait before the first word shows up. The filing's goal is to cut that delay, which it says hurts the user experience, especially on small devices that have far less computing room than a data center.
The setup is described on a system-on-a-chip (one chip holding the CPU, graphics processor, neural processor and 5G modem), hardware found inside smartphones. That fits a company that sells phone chips, since faster AI replies make a phone look better. The filing gives no before-and-after timing for the new approach, so the size of the gain stays unknown.
Qualcomm's 14th entry we've tracked in AI assistants and agents since June joins earlier work on glasses that log what you see and phones routing AI queries.
The path to a product here is short. The filing describes no new hardware, only a reshuffling of work inside the software that runs a chatbot on a phone's existing chips.
What has to exist first is a chatbot model cut into two halves that hand notes to each other, plus software willing to run them in that order. That layer is software, so it can be tested without waiting on new silicon. If the reading-first split saves time, it could reach people as a faster first word and a speedier follow-up, with no new feature to point at.
The catch is proof. The filing offers example timings for the older approaches and none for this one, and the claims are broad. Call it a sensible, modest tweak: useful if it works on your phone, and easy to miss if it does.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Qualcomm patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
13 drawing sheets from US 2026/0350403 A1 · click any drawing to enlarge
Show the other 5 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in