Qualcomm Patents an AI That Labels Every Object in a Photo Without a Preset List
Most computer vision systems can only recognize objects they were trained to look for. Qualcomm's new patent describes a pipeline that can label anything in an image, even categories it has never explicitly seen before.
How Qualcomm's open-vocabulary image labeling works
A security camera stares at an empty hallway all night. When something unusual shows up, a traditional AI might miss it entirely because the object wasn't in its training list. That's the gap Qualcomm's patent is trying to close.
The system works in two stages. First, one AI model scans the photo and draws outlines around every distinct object it finds, without needing to know what those objects are yet. Then a second model looks at each outlined region and assigns a label, pulling from an open-ended vocabulary rather than a fixed list.
The result is a segmented image where every region has a name attached to it, even if that name was never part of the original training data. For you as an end user, this could mean a phone or camera that describes a scene more accurately, without needing a software update every time a new object type comes along.
… classify each mask of the plurality of masks in accordance with an open vocabulary object classifier algorithm to obtain a label associated with each mask that identifies an associated object of the plurality of objects …
Translation: It labels every detected object using an open vocabulary instead of a fixed list.
Inside the two-model pipeline that tags image regions
The patent describes a two-stage processing pipeline for semantic segmentation (the task of dividing an image into labeled regions, one label per object or surface area).
The first stage uses an object proposal generator, a model whose only job is to find where objects are and draw masks around them. A mask here is essentially a pixel-level outline: every pixel belonging to a particular object gets flagged, and the system can crop that region out of the original image for closer inspection.
The second stage feeds those cropped regions into an open vocabulary object classifier. "Open vocabulary" means the classifier isn't limited to a hardcoded list of categories (like "cat," "car," "tree"). Instead, it can match a region to a label from a much larger, flexible set of concepts, including ones not seen during training. This is sometimes called zero-shot classification (recognizing something new without having trained on examples of it explicitly).
Finally, the pipeline stitches all the individual mask labels back together over the original image:
- Each mask gets its assigned label.
- All labeled masks are combined into one output image.
- The result is a fully segmented view where every region carries an identifying text label.
The open vocabulary object classifier model may assign labels to each segmented region and corresponding mask. The labels from all the masks are combined to produce all the masks for the original input image …
Translation: The system combines the individual labels to create a fully segmented picture.
What this means for cameras and on-device AI chips
On-device AI is moving fast, and the bottleneck is increasingly not raw power but flexibility. A system that can only recognize a fixed set of objects has to be retrained or updated every time the real world throws something new at it. Qualcomm's patent targets exactly that problem, and the company makes the chips that run AI inference inside most Android phones, many cameras, and automotive systems. A segmentation pipeline like this running efficiently on a Snapdragon chip could let a phone camera, a drone, or a security system describe unfamiliar scenes accurately without a cloud connection.
Claim 1 of this patent is written broadly enough to cover any apparatus that runs this two-stage (mask-then-label) process using an open vocabulary classifier, which means it could apply to a wide range of hardware deployments, not just phones. For context on where this sits among latest Big Tech patents in on-device vision and chip-level AI, Qualcomm is staking out foundational ground in flexible image understanding rather than a narrow product feature.
That makes this Qualcomm's 11th filing in our AI vision coverage since May, building on ideas like teaching cameras unseen objects and cutting objects via text cues.
Claim 1 covers any device that takes in an image, finds objects and draws boundaries around them, labels those objects using an open-ended vocabulary, and combines the results into a final segmented image. Nothing in the claim locks down a particular model design, chip, or industry application. That is a wide perimeter.
In practice, a claim that broad could apply to a large range of products shipping a find-then-label pipeline, from phones to cameras to cars, regardless of the underlying software choices those products make. Whether the claim survives examination at that width is the real question, because academic research in this area is extensive and examiners will have material to work with.
If it does hold, Qualcomm will own a structural description of how this category of image understanding works, which is considerably more durable than owning a single implementation.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0253436 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →