Microsoft Files Patent for an AI-Made Answer Key That Catches Chatbot Mistakes
Microsoft's filing has an AI write its own answer key from documents experts already checked, then holds chatbot replies up against that key before you see them.
What Microsoft's AI-written answer key does
Today, the usual cure for a chatbot that makes things up comes after the fact. The wrong answer has already reached you, and only later does someone notice and fix it. Microsoft's filing describes a way to check answers before they show up on your screen.
It starts with documents that human experts have already checked. An AI reads one paragraph at a time and writes a question about it, then drafts several possible answers using that paragraph. The answer that matches the paragraph most closely is kept, and the question and answer are saved together as a verified pair.
It works like an answer key the AI writes for itself. Later, when you ask a chatbot something, its reply can be compared against that key. According to the description, a reply that doesn't match can trigger a warning or the closest verified answer instead.
extracting a portion of content from a verified document; generating, using an artificial intelligence (AI) model, a prompt associated with subject matter included in the portion of content; …
Translation: The system pulls text from a trusted file and uses AI to create a question about it.
How the answer key gets built and checked
The filing's claims cover how the answer key gets built. The system starts with a verified document, meaning one a subject matter expert has checked for accuracy, and pulls out a portion of it, such as a single paragraph.
- An AI model writes a question (the "prompt") about that paragraph.
- The model then writes several candidate answers using the question and the paragraph.
- The system keeps the answer most similar to the paragraph, using a vector similarity search (text is turned into numbers that capture meaning, so passages that say similar things land close together). A cutoff score filters out weak matches.
- The question and answer are stored together as a verified pair in a database.
The description adds that the AI is run with its randomness turned down (temperature set to 0), so it sticks closely to the text. It names T5, BART and Davinci as example models for writing the questions.
Claim 7 covers the next step: when a user asks a chatbot something, the reply is checked against the stored pairs. In the description, a reply that matches goes through. One that doesn't triggers a notice or the closest verified answer instead. Users can also switch the check off when they want creative writing.
… selecting a response from the plurality of responses based on similarity to the portion of content, and storing the prompt and response as a verified prompt-response pair.
Translation: It picks the most accurate AI answer and saves the pair as a trusted reference.
Why chatbot fact-checking matters for support
If you have ever used a support chatbot, you know the risk: a confident answer that turns out to be wrong. The filing points to places where that costs more than annoyance, such as safety instructions, product assembly and medical directions. A check that runs before the reply appears could keep the worst errors from reaching you.
The filing is also open about limits. A reply can only be checked if the answer key holds a similar question, and otherwise the system tells you it can't verify the answer. The key also covers only what the verified documents say. The likely fit is a company with a large, trusted pile of documents, like a help center or a manual library.
This is the 25th Microsoft filing we've tracked since July in the AI guardrails race, following one that re-checks its own answers and one tracing email inspiration.
The problem here is large and costly. Chatbots state wrong things in the same confident tone as right things, and in customer support, safety guidance or health questions, a wrong answer can hurt someone. Hiring experts to write a fact list for every new document cannot keep pace with how fast content piles up, so having a machine draft the list is a sensible match for the size of the job.
The weak spot is that an AI is grading an AI. The safeguard is that every answer in the key must closely match a paragraph experts already approved, so the key stays tied to a trusted source. Still, closeness in wording is a rough stand-in for truth, and the filing admits that questions with no similar match in the key cannot be checked at all.
So the approach fits narrow, document-heavy settings well, such as a company answering questions from its own manuals. It looks too thin to fix chatbot errors in general, and the filing never claims it would.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0340280 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in