Qualcomm Patents a Way to Trim Down the Visual Data AI Has to Read in Any Video
Ask an AI what happened in a video, and it has to process every single frame first. Qualcomm's new patent describes a way to let the AI skip most of that data without missing the point.
How Qualcomm wants your phone's AI to handle video questions faster
Today, when an AI tries to understand a video, it typically crunches through every frame one by one. That is a lot of work, and on a phone or a lightweight chip, it can be slow or simply impossible.
Qualcomm's filing describes a system that converts each video frame into a compact data package called a visual token, then picks only the most relevant ones before handing anything to the AI. Rather than feeding the whole video into a large language model, the system first figures out what you are asking, then selects the slice of frames that actually answers the question.
The result is that the AI sees a much smaller, curated set of information instead of the full flood of frames. That should make video question-answering faster and more practical on everyday devices like phones and tablets.
encode each frame of a plurality of frames of video data to generate a plurality of visual tokens; select a subset of visual tokens from among the plurality of visual tokens; …
Translation: The system turns video frames into digital pieces and then throws away the unneeded ones.
How the system picks which video frames' data the AI actually reads
The patent describes a four-step pipeline running on a single device.
First, a visual encoder (a specialized image-reading model) converts every frame of a video into a visual token, a compact numerical fingerprint that represents what is in that frame. A one-minute clip at 30 frames per second produces 1,800 of these fingerprints, which is far too many for a phone-class AI to handle all at once.
Second, the system selects a subset of those tokens. The patent does not lock in a single selection method but the core idea is that not all frames matter equally for any given question, so only the useful ones move forward.
Third, your query (the question you typed or spoke) is converted into a text embedding, a numerical representation of your words that the AI can compare against visual data.
Finally, the pruned set of visual tokens and the text embedding are fed together into a large language model (an AI that reads and writes text, similar to what powers chatbots) which generates an answer. The design is explicitly built for on-device hardware, pointing to Qualcomm's own mobile and edge chips.
What this means for AI assistants running on phones and chips
For you as a user, the payoff is an AI assistant that can answer questions about a video clip without needing to send that video to a remote server. Processing happens on the device, which is faster, private, and works without a strong internet connection.
Qualcomm's steady investment in on-device AI inference makes this filing fit a clear pattern: push capable AI onto the chips it already sells for phones and cars. If a method like this works well in practice, it could make video-aware AI assistants a standard feature on mid-range Android devices rather than a cloud-only premium.
This is the 44th Qualcomm filing we've tracked in the AI chip wars since July, adding to work like one on overheating phones and one on shared bandwidth.
Claim 1 covers any system that breaks a video into frames, turns those frames into numbers, picks a smaller batch of those numbers, turns a question into numbers, and feeds everything into a large language model to get an answer. The claim puts no restrictions on how the selection happens, what ratio of frames must be kept, or what type of encoder does the converting.
That scope is enormous. Those four steps describe the obvious sequence any engineer would follow when building a "ask me anything about this video" feature, and the claim would hand Qualcomm a legal toll booth over all of it.
The filing's real story is about running this kind of video reasoning directly on a phone or laptop, without sending footage to a remote server. The broad claim amplifies that story, but patent examiners will almost certainly force it to narrow before anything is granted.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
16 drawing sheets from US 2026/0270448 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →