Qualcomm Patents a Way to Ask an On-Device AI About Multiple Parts of a Photo at Once
Most phone AI can only focus on one area of a photo at a time. Qualcomm's new patent describes a system that crops out every region you care about, tiles them into a single picture, and feeds that combined image to an AI model in one shot.
How Qualcomm's multi-region image AI actually works
Imagine you take a photo of a restaurant menu sitting next to a dish, and you want to ask your phone's AI: "What's in this dish, and how much does it cost?" Today, most AI systems on phones handle that kind of question awkwardly because they process the whole image as one blob, losing the fine detail in the text, or they can only zoom in on one area at a time.
Qualcomm's patent describes a different approach. You (or an app) mark out the parts of the image that matter, like the price on the menu and the plate in front of you. The system then cuts those regions out and arranges them side by side on a single canvas image, like a collage. That collage goes into the AI as a single input, so the model sees all the relevant parts together and can answer questions about the whole scene.
The key benefit is that this happens on the device itself, using Qualcomm's chips, without sending your photo to a server. That matters for speed, privacy, and working without a reliable internet connection.
… arrange image representations of the ROIs into a single canvas image; input the canvas image into the image encoder to obtain image tokens associated with cells of the canvas image …
Translation: The device stitches different parts of a photo into one image so the AI can process them all at the same time.
How the canvas tiling feeds the image encoder
The patent describes a device (think a phone or tablet running a Qualcomm chip) that pairs two AI components: an image encoder, which converts visual information into numerical tokens the AI can read, and a large language model (LLM), which generates text responses based on those tokens.
Here is the step-by-step flow the patent outlines:
- The device receives an image and a set of regions of interest (ROIs), specific rectangular or bounded areas inside that image that the user or app has flagged as important. At least one ROI must contain text.
- The system crops each ROI and arranges them all into a single canvas image, a tiled collage. This is the core trick: instead of running the encoder multiple times (once per region) or sending the full high-resolution original, everything gets packed into one standardized input.
- The canvas goes through the image encoder, which produces image tokens tied to cells of the canvas grid. Each cell corresponds to one of the original ROIs.
- Those tokens, along with the user's query, go into the LLM, which generates a natural-language answer.
The approach is designed to handle images where both visual content (objects, people, scenes) and text content (labels, prices, signs, captions) appear in different parts of the frame simultaneously, without forcing the model to choose which part to focus on.
The one or more processors are configured to obtain data representing multiple regions of interest (ROIs) within the image, the multiple ROIs including one or more ROIs that include text.
Translation: The system identifies specific areas of a photo to focus on, including any parts that contain written words.
What this means for AI running on your phone's chip
For anyone with a recent Android phone running a Snapdragon chip, this patent describes the kind of AI that could power a camera assistant that reads a receipt, identifies items, and answers questions about it in one tap, entirely offline. The canvas-tiling approach is efficient enough to run inside the tight memory and power limits of a mobile chip, which is something cloud-dependent AI tools sidestep by just shipping the work to a data center.
Qualcomm's position in the market is specific: the company makes the processors inside most premium Android phones, and it has been pushing on-device AI hard as a selling point for its Snapdragon series. This patent fits squarely into that strategy, and it is part of the broader wave of new tech patents around on-device multimodal AI from chip companies racing to run large models locally rather than in the cloud.
Qualcomm's second AI recommendation patent we've tracked since May follows one fusing GPS with road maps.
Packing multiple image crops into a single collage before analysis saves real processing power, but it forces every crop to share the same fixed canvas space. The more regions you add, the smaller each one gets, and a price tag or street sign shrunk to a thumbnail may become harder to read than if the phone had never cropped it at all.
The patent does not describe a way to give text-heavy regions more canvas space than, say, a background object. That gap matters most in exactly the scenes this feature seems built for: menus, receipts, dense signage, anywhere small print is the whole point.
The compute savings are structural and real, especially on a phone with strict battery limits. But the design trades resolution for efficiency in a way that could undermine accuracy, and whether that trade holds up depends on engineering choices this document leaves open.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
13 drawing sheets from US 2026/0253398 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →