Get the newsletter
Microsoft · AI / ML · Filed Jun 8, 2026 · USPTO published Oct 1, 2026 · Oct 1 release

Microsoft Patents an Image Search That Looks for Meaning, Not Just Pixels

Most image search tools look for visually similar pictures, matching colors, shapes, and textures. Microsoft's new patent describes a system that skips all that and asks a different question: do these two images mean the same thing?

A user interface displays a geometric problem image and its detailed solution steps, related videos, and concepts. Drawing from patent filing US 2026/0300384 A1.
A user interface displays a geometric problem image and its detailed solution steps, related videos, and concepts.All 12 drawings ↓
Publication number
US 2026/0300384 A1(US20260300384A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Jun 8, 2026 · Oct 1, 2026
Inventors
Manish GUPTA, Niraj Nrisinvha BHILEGAONKAR, Avishek MAZUMDER, Akash KATARE
Status when we published
Waiting for an examiner
Document
20 claims
US classification
707/741
Parent application
Continuation of 18/970,647 (filed Dec 5, 2024)(18970647)

What Microsoft's meaning-first image search actually does

Imagine you take a photo of a busy street intersection with a traffic light on the left and pedestrians crossing on the right. You want to find other images that capture that same scene setup, not images that look identical, but images where the relationships between things are the same. A standard search engine would struggle with that. Microsoft's patent describes a system that can do exactly this.

Instead of comparing raw pixels or colors, the system analyzes what elements are in an image and how they relate to each other. It encodes that relational structure into a compact form, stores it in a database, and when you drop in a query image, it finds stored images that share the same underlying structure of meaning.

The result is a search tool that can recognize, say, that a photo of a dog sitting next to a child is meaningfully similar to a cartoon drawing of the same setup, even if the two images look nothing alike at first glance.

From the filing · CLAIM 1
… extracting, using one or more machine-learned models, semantic structure information from the query image, the semantic structure information representing relationships among elements depicted in the query image …

Translation: AI is used to figure out how the different objects in your photo relate to one another.

How the system extracts and compares image meaning

The core of the patent is what Microsoft calls semantic structure information: a machine-generated description of the relationships between objects in an image, not just a list of what's there, but a map of how things connect.

Here's how the process works:

  • A machine-learned model (an AI trained to understand image content) scans each image and pulls out its structural meaning. Think of it as generating a relational blueprint rather than a pixel fingerprint.
  • Those blueprints are stored in an index (a fast-lookup database optimized for search).
  • When you submit a query image, the same extraction process runs on it, and the system compares your image's blueprint against the stored ones.
  • Images whose relational structure closely matches yours are returned as results.

The claim is specifically about relationships among elements, not just element detection. Two images could contain dogs and children but arrange them differently, and the system would treat them as distinct. Two images could look very different but share the same spatial and semantic relationships, and the system would flag them as similar.

The patent does not specify a single model architecture, leaving room for various approaches, which suggests the inventors are staking out the broader method rather than one specific implementation.

From the filing · THE ABSTRACT
… an image search is performed based on a query image where the result includes images with semantic information that matches the query image …

Translation: The system finds other pictures that share the same underlying meaning and context as your search image.

What this means for finding images in search and at work

For everyday users, this kind of search could make tools like Microsoft's Bing Image Search or OneDrive photo organization far more useful. Right now, searching images by meaning requires typing text descriptions. With a system like this, you could drop in an image and find conceptually equivalent photos across huge libraries, without having to describe what you're looking for.

For businesses, the implications go further. Design teams, stock photo platforms, legal discovery, and media archives all deal with the problem of finding images that capture a specific situation rather than a specific look. Microsoft's run of AI-powered search filings points to a company building out the infrastructure to make this kind of contextual retrieval a standard capability, not a specialty tool.

Microsoft filed its 71st application in Language AI we've tracked since May, adding to work like one grouping cloud alerts and one flagging document gaps.

Source: US 2026/0300384 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 12 sheets

12 drawing sheets from US 2026/0300384 A1 · click any drawing to enlarge

Image search system overview., drawing sheet 1 from US 2026/0300384 A1
Sheet 1 Image search system overview.
Neural network training data., drawing sheet 2 from US 2026/0300384 A1
Sheet 2 Neural network training data.
Image similarity reasoning logic., drawing sheet 3 from US 2026/0300384 A1
Sheet 3 Image similarity reasoning logic.
Offline and online image processing., drawing sheet 4 from US 2026/0300384 A1
Sheet 4 Offline and online image processing.
Geometric problem solution interface., drawing sheet 5 from US 2026/0300384 A1
Sheet 5 Geometric problem solution interface.
Algebraic problem solution interface., drawing sheet 6 from US 2026/0300384 A1
Sheet 6 Algebraic problem solution interface.
Training data generation process., drawing sheet 7 from US 2026/0300384 A1
Sheet 7 Training data generation process.
Image similarity determination process., drawing sheet 8 from US 2026/0300384 A1
Sheet 8 Image similarity determination process.
Show the other 4 sheets
Microsoft Patents an Image Search That Looks for Meaning, Not Just Pixels patent drawing, sheet 9
Sheet 9
Microsoft Patents an Image Search That Looks for Meaning, Not Just Pixels patent drawing, sheet 10
Sheet 10
Microsoft Patents an Image Search That Looks for Meaning, Not Just Pixels patent drawing, sheet 11
Sheet 11
Microsoft Patents an Image Search That Looks for Meaning, Not Just Pixels patent drawing, sheet 12
Sheet 12
Patent filing page

Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.