Microsoft Patents Technology to Catch AI Errors Before They Cause Damage
AI chatbots sometimes confidently state things that are simply wrong. Microsoft has filed a patent for a system that forces an AI to show its work so a separate process can check whether the answer is actually correct.
How Microsoft's AI answer-checker works for regular people
You're staring at a spreadsheet summary your AI assistant just produced, wondering if those numbers are real or something the AI invented. That's the hallucination problem: AI models sometimes generate confident-sounding answers that don't match the source material at all.
Microsoft's patent describes a system that slips a hidden instruction into every question sent to an AI. That instruction tells the AI not just to give an answer, but to also write out exactly where each number or fact came from and a small piece of runnable code that re-derives the answer from the original data. A second system then actually runs that code and checks whether the result matches what the AI claimed.
If the numbers match, the answer is likely trustworthy. If they don't, you know something went wrong. The goal is to give you a way to verify AI output automatically, without needing to read every source document yourself.
automatically injecting, into a prompt including query text and input data, a verification request for a query result to be derived from the input data …
Translation: The system adds a request for proof directly into the prompt sent to the AI.
Inside Microsoft's prompt-injection verification chain
The patent describes two linked processes. The first automatically injects a verification request into the prompt (the message you or an app sends to an AI model). The injection happens without the end user needing to do anything.
That verification request instructs the AI to produce three things alongside its normal answer:
- The specific value it is asserting (a number, a date, a name)
- The source within the input data from which that value was drawn (a cell, a paragraph, a row)
- An evaluable expression, meaning a short code snippet written in a machine-readable language, that can be run independently to re-derive the value from the source
The second process receives all of that output, runs the code snippet, and compares what the code actually produces with what the AI claimed. If they match, the answer is considered validly derived. If they don't match, the system flags the result as a potential hallucination.
The key technical move here is turning a subjective "is this right?" question into an objective, runnable test. Rather than asking a human or a second AI to judge the answer, the system uses deterministic code execution (code that gives the same result every time) to check the AI's reasoning.
… cause the LLM to generate verification data that indicates a derivation of the query result from the input data …
Translation: It forces the AI to show its work and explain how it reached its answers.
What this means for trusting AI at work
For anyone using AI to summarize contracts, financial reports, or any document where getting a number wrong has real consequences, this kind of automatic fact-checking is a big deal. Right now, catching hallucinations usually means manually cross-referencing sources, which defeats much of the time-saving point of using AI in the first place.
Microsoft's steady investment in AI reliability tooling shows up here in a practical form. If this system works at scale, it could make AI-generated summaries in products like Microsoft 365 Copilot far easier to trust, because every answer would carry its own built-in audit trail that a machine already checked.
That makes this Microsoft's 26th filing we've tracked in our AI models working in teams watchlist since May, following one on picking the right cloud model and one on a three-model photo caption system.
Claim 1 is broader than it might first appear. It covers any system that injects a verification request into a prompt, asks the AI to produce a source reference and a runnable code snippet, and then checks the output by running that code. That combination is specific enough to describe a real engineering approach, but it does not tie down to any particular AI model, document type, or coding language.
That breadth matters because it could apply to almost any scenario where an AI answers questions about provided data, from financial analysis tools to legal document review to enterprise search. A granted patent here would give Microsoft a claim over a fairly fundamental quality-assurance pattern for data-grounded AI responses.
The honest caveat is that this approach only works when the AI's answer is supposed to come from structured or semi-structured input data. It does not help when an AI is generating creative text or answering from general knowledge. Within its lane, though, this is one of the more credible technical approaches to hallucination detection filed so far, because it replaces opinion-based checking with code execution.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0267900 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →