Qualcomm Patents a Two-Model AI System That Verifies Its Own Output
Qualcomm's new patent describes a way to make AI text generation faster by having a small, cheap model write a draft answer, then sending only the most uncertain parts to a larger model for a second opinion. It's a smarter division of labor than running the big model for every single word.
How Qualcomm's draft-and-verify AI generation works
Imagine you're proofreading a document. Instead of re-reading every word yourself, you skim it first and only hand the tricky sentences to a specialist. That's essentially what Qualcomm is patenting here.
When an AI generates text, it normally produces one word (or "token") at a time, which is slow. Speculative decoding is a technique where a small, fast model drafts a batch of words all at once, and a larger, more accurate model checks whether those drafts are good enough to keep. Qualcomm's twist is being selective about which draft words get sent for checking, rather than sending the whole batch every time.
The result is that the big model does less work, the small model carries more of the load, and you get your answer faster. This is especially relevant for AI running directly on a phone or laptop chip, where power and processing speed are limited.
generate, using a secondary generative machine learning model associated with a primary generative machine learning model, a first set of tokens; select a first subset of tokens, from the first set of tokens, for verification by the primary generative machine learning model; …
Translation: A smaller AI model creates an initial draft and hands parts of it over to a larger AI model to check.
Inside the token selection and verification loop
The patent describes a hybrid speculative decoding system built around two machine learning models working in tandem.
First, a secondary (draft) model generates a full set of candidate tokens. Tokens are the smallest units of text an AI works with, roughly word fragments. This draft model is small and fast but less accurate than the main model.
Next, the system selects a subset of those tokens for verification. Not all draft tokens are sent to the larger model; only the ones that need a second opinion, presumably those the draft model was least confident about. The patent does not fully detail the selection criteria in the claim, leaving room for various scoring strategies.
The primary (main) model then reviews only that subset. Tokens it approves are kept as verified; the rest of the original draft tokens that were never sent for checking are included in the final output as-is.
The output is assembled from:
- Tokens the primary model verified and accepted
- Remaining draft tokens that bypassed verification
This pipeline reduces how often the large model is invoked, which cuts latency (the delay between asking a question and getting an answer) and reduces power consumption.
… a machine learning model output comprising one or more verified tokens from the subset of tokens and one or more remaining tokens from the set of tokens is generated.
Translation: The final response combines the checked pieces and the unchecked pieces into one unified output.
What this means for on-device AI speed
For on-device AI, every computation costs battery life and time. Right now, running a capable large language model on a phone chip means either slow responses or a heavily compressed model that sacrifices quality. Qualcomm's approach tries to get close to large-model quality at small-model speed by being selective about when the big model actually does any work.
Qualcomm makes the chips inside most Android phones and many laptops. A patent like this points toward the kind of AI inference optimization you'd expect to see baked into future Snapdragon processors. If it works well in practice, your phone could generate longer, more accurate AI responses without draining the battery any faster than today's simpler models do.
Speculative decoding is already an established technique in AI research, so this patent is really about Qualcomm's specific implementation: selectively filtering which draft tokens get verified rather than batch-checking everything. It's an incremental engineering refinement, not a new idea from scratch. Whether the selection logic is novel enough to matter in practice depends on details the patent's abstract and claim don't fully reveal, but on-device AI inference is exactly the battleground where Qualcomm needs every efficiency edge it can get.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0228513 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in