Google Files Patent for Image Search That Guesses Where Your Taste Is Heading
Search with a photo, tap a result you like, and Google's patent application has the system guess where your taste is heading and search there, a step beyond what you tapped.
What Google's two-image search actually does
You snap a photo of a chair in a waiting room, search with it, and get back pictures of the waiting room. You scroll, tap a result that is a little closer to what you want, and the search mostly shows you more of the same.
Google's patent application describes a fix. When you tap a result, the system treats it as a clue instead of a fresh start. It looks at your original photo and the one you tapped, works out how they differ, and then keeps going in that direction, searching for what lies a step beyond your pick.
You can also tell it what you don't want. Tap an image that has the detail you dislike, and the results look for your original object without that detail. You can crop to the part of a picture that matters, too.
… determining a third embedding based on a projection prediction associated with an embedding change from the first embedding to the second embedding; determining a second set of search results based on the third embedding; and providing the second set of search results for display in the search results interface.
Translation: The system predicts your future aesthetic direction and immediately surfaces new images matching that trajectory.
How Google projects your next search forward
The claims cover a two-step search. First, you search with one image and get a page of image results. Then you pick one of those results, and the system uses both pictures to run a second search.
The key step is how it combines them. An embedding model (software trained to turn a picture into a long list of numbers describing what it looks like) converts both images into numbers in the same embedding space (think of a huge map where similar-looking things sit close together). The system measures the shift from your first image to the one you picked, then produces a third point by continuing that shift past your pick. The second search looks around that third point.
The claims and the description add several variations:
- The shift has a size and a direction, and the projection can be scaled (for example, to half the distance) based on context or trend data.
- Picking a result can mean 'show me this without that detail,' and the results are pruned to match.
- You can crop the first image or select part of the second, and the filing describes outlining objects with segmentation masks (automatic cutouts) before searching.
- Weights can depend on context, such as when you looked at each image.
Systems and methods for multi-image search can include obtaining two or more images and determining one or more search results that are based on the two or more images.
Translation: The technology looks at multiple pictures you select to figure out what you are looking for.
What this means for shopping and inspiration searches
If you've ever searched for a couch, a dress or a patterned tile and gotten the same room over and over, this is aimed at you. Searches built on pictures are hard to steer with words, and this lets a tap do the steering. The filing says better results could mean fewer searches per session, which also saves computing work on Google's end.
The same idea could suit shopping, home design and fashion, where people recognize a look when they see it and struggle to describe it. The filing also describes running a search automatically on images you've saved, liked or lingered on, and offering an explore button inside a collection of saved images.
Google's 108th filing in the Language AI patents we cover since May follows a faster text AI shortcut and an ad ranking shortcut.
Nothing in this filing needs new hardware. The text says the work can run on a phone, a laptop or a server, and the ingredients are an image-matching model and a results page that any visual search tool already has.
The shortest route to a product is a small tap-to-refine step. The filing sketches the screen: thumbnails of both images in the search box and fresh results in place of the old ones.
What still needs work is the guessing. Following your taste a full step further can overshoot, so the filing also describes taking a half step or adjusting the distance with context, such as which image you touched more recently. Tuning that dial decides whether it feels helpful or presumptuous.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
14 drawing sheets from US 2026/0320109 A1 · click any drawing to enlarge
Show the other 6 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in