Get the newsletter
Microsoft · AI / ML · Filed Mar 28, 2025 · USPTO published Oct 1, 2026 · Oct 1 release

Microsoft Patents a System That Reads Your Screen and Answers Questions You Didn't Ask

Most people ignore the AI features baked into their laptops because using them requires effort. Microsoft's new patent describes a system that skips the asking entirely, reading what's on your screen and surfacing relevant context the moment you need it.

A web browser displays a recipe with suggested follow-up questions and answers based on the page's content. Drawing from patent filing US 2026/0299972 A1.
A web browser displays a recipe with suggested follow-up questions and answers based on the page's content.All 6 drawings ↓
Publication number
US 2026/0299972 A1(US20260299972A1)
Applicant
MICROSOFT TECHNOLOGY LICENSING, LLC
Filed · Published
Mar 28, 2025 · Oct 1, 2026
Inventors
Medhaj Suresh ATHILKAR, Yan YAN, Benjamin Bear STOLOVITZ, Bryce Allen HENTHORN, Melinda IVANOV, John Joseph BUGLIONE III
Status when we published
Waiting for an examiner
Document
20 claims
US classification
715/764

What Microsoft's automatic screen-reading AI actually does

Every time you pause on a paragraph you don't quite understand, or hover over a chart that needs more background, that hesitation is a small friction point. Most AI tools in that moment require you to copy the text, open a chat box, type a question, and wait. That's enough steps to make many people give up.

Microsoft's patent describes a system that watches for those moments automatically. When you do something that signals you might want more information, like pausing, hovering, or highlighting, it captures what's on your screen, figures out what the content is about, and then generates a relevant follow-up question plus an answer, all without you lifting a finger.

The result shows up in a section of your interface as a ready-made question and response. You didn't have to ask; the system noticed you might need it and did the work. It can also pull in extra information tied specifically to your account or context, so the answer is more relevant to you personally.

From the filing · CLAIM 1
… determining, by the computational model, a follow-up input based on the semantic content of the content capture; retrieving, by the computational model supplemental information from a supplemental information repository; determining, by the computational model, an output corresponding to the follow-up input …

Translation: The system figures out what you might want to know next and automatically looks up the answers.

How the system captures content and decides what to explain

The patent describes a pipeline that starts with a user activity trigger, a detectable action like hovering, pausing, or highlighting text or images on screen. When that trigger fires, the system takes a content capture, essentially a snapshot of the relevant on-screen visual content.

That capture is sent to a computational model (the patent's term for what is effectively a large language model, the same category of AI behind tools like ChatGPT). The model does two things in sequence:

  • It identifies the semantic content of the capture, meaning the underlying concepts or ideas, not just the raw words or pixels.
  • It formulates a follow-up input, a question or prompt that a user would logically want answered given that content.

The model then retrieves supplemental information from a repository, which can be tied to the specific user entity (your account, your organization, your prior context). It uses that supplemental data to generate an output corresponding to the follow-up question. Both the question and the answer are rendered visibly in a section of the graphical user interface.

The key structural point is that the system determines what question to ask on your behalf. You don't type anything. The trigger is behavioral, the question is generated, and the answer is pre-loaded.

From the filing · THE ABSTRACT
… the present system proactively extracts on-screen content via a content capture for analysis by a computational model (e.g., a generative language model) in response to a user activity trigger.

Translation: Your device takes screenshots on its own and runs them through an AI whenever you do something.

What this means for everyday Windows and Office users

For Windows and Office users, this kind of system could show up anywhere you read or view content: a Word document, a PDF in Edge, a slide deck, a spreadsheet. The promise is that you'd see a relevant explainer or follow-up automatically, trimming the back-and-forth of opening a separate AI chat window.

The practical friction here is consent and distraction. A system that proactively pops up information every time you pause risks feeling intrusive. the pattern in Microsoft's AI assistant filings points toward deeper, more ambient AI integration across Windows, and this patent fits that direction. Whether users actually want their screen watched that closely is a product question the patent doesn't answer.

Microsoft's 18th filing we've tracked since July on our AI agents working for you watchlist follows one that deploys network defenses automatically and one building agents from plain English.

Source: US 2026/0299972 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 6 sheets

6 drawing sheets from US 2026/0299972 A1 · click any drawing to enlarge

Web page with suggested questions., drawing sheet 1 from US 2026/0299972 A1
Sheet 1 Web page with suggested questions.
Email with suggested locations., drawing sheet 2 from US 2026/0299972 A1
Sheet 2 Email with suggested locations.
Text messages with flight details., drawing sheet 3 from US 2026/0299972 A1
Sheet 3 Text messages with flight details.
System for content analysis., drawing sheet 4 from US 2026/0299972 A1
Sheet 4 System for content analysis.
Steps for content analysis., drawing sheet 5 from US 2026/0299972 A1
Sheet 5 Steps for content analysis.
Computer system components., drawing sheet 6 from US 2026/0299972 A1
Sheet 6 Computer system components.
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.