Qualcomm Patents a Way to Skip Repetitive Video Frames Before AI Watches
Qualcomm's patent decides which video frames an AI should bother with by comparing two sentences: one describing the whole clip, one describing a single frame. If they say about the same thing, the frame gets skipped.
What Qualcomm's caption-comparing video filter does
Imagine you ask your phone's AI assistant, "What happened in this 30-second clip?" The clip has 900 frames, and most of them show a person crouching in nearly the same pose. A typical AI would chew through every one of those frames, which is slow and burns battery for very little new information.
Qualcomm's filing describes a shortcut. First, the system writes a short caption for the whole clip, such as "a man rises from a crouch." Then it writes a quick caption for individual frames. If a frame's caption says roughly the same thing as the clip's caption, the frame is redundant and gets left out.
Frames that say something different, like "a man is running," are kept, and the more different they are, the more of their detail the AI gets to see. The point is to feed the AI less, but more useful, material.
… determine to process zero or more visual tokens selected from among the second plurality of visual tokens based on a similarity between the first text and the second text …
Translation: Deciding whether to skip parts of a new video frame by comparing text descriptions of the video.
How two captions decide which frames get dropped
The patent covers a method for deciding how much of a video a large AI model actually needs to look at. Video AI models (often called large multimodal models, meaning they handle pictures and text together) break each image into visual tokens, small chunks of numbers that stand in for what the image shows. One frame can produce hundreds or thousands of tokens, so a video can reach millions, which the filing calls a performance bottleneck.
Here is the flow the claims describe:
- A video encoder turns the clip into one set of tokens, and a video-captioning model turns those into a text description of the whole clip.
- An image encoder turns individual frames into their own tokens, and an image-captioning model writes a caption for each frame.
- The system measures how similar each frame caption is to the clip caption. The filing uses perplexity, a score for how surprised a language model is by a piece of text.
- If a frame's caption is too similar (the score does not pass a threshold), zero of its tokens are used. Otherwise some are kept, and the AI processes the clip tokens plus the chosen frame tokens.
The description gives a worked example: scores of 0.4, 0.4 and 2.4 for "crouching," "crouching" and "running." It says the running frame could contribute six times more tokens than a crouching one. It also notes you could check only every tenth or 30th frame.
… processing the first plurality of visual tokens to generate a first text, wherein the first text is descriptive of the video data …
Translation: Converting visual elements of the video into a descriptive text summary.
Why this matters for AI video on phones and cars
If an AI can skip frames that add nothing, answering questions about your videos gets cheaper to run. That matters most on devices with limited power, and the filing lists phones, wearables, AR and VR headsets, vehicles and robots as places the method could run. Qualcomm makes chips for many of those, so efficiency tricks like this fit its business.
The filing also argues that comparing captions picks frames that matter for the meaning of a scene. Comparing raw image data, by contrast, can favor frames that merely look different. For questions like "what did the person do?", that choice is the useful one. Older approaches fixed the number of tokens in advance, while this one adjusts to each video.
Qualcomm's seventh entry in our AI image and video coverage since May follows earlier applications on filling in missing photo areas and fewer steps in AI video.
On the ship-path question, this one sits close to the finish line. The claims describe software steps only: encoders, captioning models, a similarity score and a threshold. There is no new chip or sensor required, and the filing says it can run on a phone, a watch, a headset or a car.
What has to exist first is a video AI model running on the device, plus two captioning models to do the comparing. That adds work of its own, since writing captions for frames costs computing time. The filing does not say how much that overhead eats into the savings, so the real-world gain is an open question.
The shortest route to a product is a feature inside a video-understanding app trimming frames before the main model runs. It reads as a practical efficiency fix rather than a showpiece, and that makes it the sort of idea that could get used without anyone noticing.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Qualcomm patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0360166 A1 · click any drawing to enlarge
Show the other 2 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in