Google Files Patent for Teaching Image Search Which Object You Mean
Google's patent teaches search to find the dog in your photo by training on descriptions that never say 'dog.' A language model writes clues like 'the thing with four legs,' and only clues that fit exactly one object survive.

- Publication number
- US 2026/0369023 A1(US20260369023A1)
- Applicant
- Google LLC
- Filed · Published
- May 27, 2025 · Nov 19, 2026
- Inventors
- Lav Singh, Devansh Gautam, Aishwarya Mittal, Inderpreet Singh Saggu, Manish Arora, Nitish Krishna Murthy
- Status when we published
- Examiner raised objections; the company can respond
- Document
- 20 claims
- US classification
- 382/103
- Examiner
- KAUR, JASPREET (Art Unit 2662)
What Google's photo-search training trick does
Ever snapped a photo of a crowded scene and typed a question about just one thing in it, like 'how to bleach?' when you meant the hair, not the shirt? Search has to guess which part of the picture you mean. The filing says existing systems can misidentify that part, or simply process the whole image.
Google's patent is about making that guess better, and the trick is in the practice material. A language model looks at a photo where one object, say a dog, is already boxed. It then writes descriptions of the dog that never use the word 'dog,' such as 'the animal' or 'the thing with four legs.' Any description that could also fit something else in the picture, like a duck, gets thrown out.
The photos and surviving descriptions then train a model to pick the right spot in a picture when you type a vague question, so the search can focus on that part of your image.
… each of the plurality of synthetic text queries referencing the particular object depicted in the image without using a term from the object label; …
Translation: The system writes search phrases for an object without using its actual name.
How Google writes clues that never name the object
The filing covers two linked pieces. The first builds training material. The system starts with an image, a label for one object in it (say 'dog'), and a ground truth bounding box, a rectangle marking where that object sits, treated as the correct answer. A generative language model, such as a vision language model that reads images as well as text, writes several descriptions of that object without using any word from the label.
The filing's example is a yard scene with people, a dog, a duck, a house and trees. Candidate clues include 'the standing thing,' 'the animal' and 'the thing with four legs.' Each gets scored for whether it could fit some other object, and only 'the thing with four legs' survives.
The surviving clue, image and box become one training example. A multimodal region-of-interest model (software that links words to a region of a picture) scores candidate boxes against the clue, and a loss function (a measure of how far off it was) nudges it toward the right box.
The other claim group covers the search side: a photo plus typed words goes in, an object detector draws boxes around everything, the model scores each box against the words, and the search engine runs on the best match.
The computing system includes multimodal region-of-interest (ROI) model, which is trained with the training example to determine one or more particular bounding boxes associated with the particular object depicted in the image.
Translation: An AI model learns to draw accurate boxes around specific items in photos.
Why this matters for Google's photo-plus-text search
If you search with a photo and a few words, you expect the search to look where you mean. The filing says current systems can misidentify the region or process the entire image, which hurts the relevance of results. A better box-picker could mean fewer follow-up searches, and cropping to the right object can also shrink what gets sent to the search engine.
The patent also leans on cheap training data: a language model writes the practice descriptions, which the filing says cuts the computing needed to gather and store training data. The examiner's first review rejected the claims as filed, which is routine and usually followed by amendments, so the final scope may be narrower. Nothing here says when, or whether, you would see it in a product.
This is the 54th Google patent we've tracked in our AI vision coverage since May, adding to work like one that scores its own guesses and one borrowing faces for group photos.
Source: US 2026/0369023 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 22 sheets
22 drawing sheets from US 2026/0369023 A1 · click any drawing to enlarge
Show the other 14 sheets

No comments yet