Get the newsletter
Microsoft · AI / ML · Filed May 16, 2025 · USPTO published Nov 19, 2026 · This week's release

Microsoft Patents a Grading Checklist That Rewrites Itself When AI Misses Something

Grading an AI's answers against a fixed list of questions misses any mistake the list's authors never thought of. Microsoft's patent lets the AI grader jot down notes about what the list missed, then turns repeated notes into new questions.

An AI spreadsheet assistant answers questions and writes formulas inside a table editor. Drawing from patent filing US 2026/0368214 A1.
An AI spreadsheet assistant answers questions and writes formulas inside a table editor.All 9 drawings ↓
Publication number
US 2026/0368214 A1(US20260368214A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
May 16, 2025 · Nov 19, 2026
Inventors
Sumit GULWANI, Mukul SINGH, Mansi UNIYAL, Vu Minh LE, Gust Ben Anneloes VERBRUGGEN
Status when we published
Waiting for an examiner
Document
20 claims
US classification
704/9
Examiner
RILEY, MARCUS T (Art Unit 2654)

What Microsoft's self-updating AI checklist does

You ask the AI assistant in your spreadsheet to explain a formula, and the answer reads fine. Someone at the company still has to decide whether it is actually good. That means grading thousands of answers against a list of questions like "Is it clear?" and "Is it accurate?"

Here is the catch: a fixed list only catches the mistakes somebody thought of in advance. Microsoft's patent adds a margin note. The AI grader answers every question on the list, and it can also write a free-form comment about anything the list missed, like a typo in your original request.

When the same kind of note keeps showing up, the system turns it into a brand-new question and adds it to the list. The checklist grows along with the problems it finds.

How a free-form note becomes a new question

The system starts by building one prompt (the written instructions sent to an AI) that bundles two things: a piece of AI-written content to be judged, and a checklist of predefined questions about it. The checklist also gives the grading AI an option to write a free-form response, which is just a comment in its own words.

The grader sends back its answer to every checklist question. If it spots something the questions don't cover, it adds a comment too. If it has nothing extra to say, it adds none. The filing's own example: a spreadsheet request misspells "highlight" as "heighlight." Asked whether the request avoids ambiguous language, a grader with no outlet might wrongly say no. With the comment option, it can answer yes and note the typo separately.

Then the checklist gets updated:

  • Gather free-form comments from many evaluations and find the recurring ones.
  • Generate several candidate questions from them and pick the best.
  • Prune questions that are redundant or low value.

The loop can repeat as new content arrives. The filing describes fully automatic updates, human approval of each change, or a mix. The claims center on spreadsheet explanations, such as explaining a formula, though the description says the same idea could grade code, translations, and more.

Why spreadsheet AI answers need better grading

If you use an AI assistant inside an app, someone behind the scenes is grading its answers before the product ships and after. A checklist that updates itself could catch new kinds of mistakes sooner, and it cuts the manual work of writing and maintaining those grading lists. You would never see this tool directly, but you might notice the result in clearer, more complete answers.

The filing's examples come from spreadsheets, where a vague or incomplete formula explanation is easy to spot. The description says the same loop could grade medical summaries, legal analyses, customer service replies, and code explanations. That breadth matters because every company shipping an AI assistant has the same grading headache.

This is the 34th Microsoft filing we've tracked since July in the AI guardrails race, following one on blocking AI from company folders and one on an AI team rewriting chatbot instructions.

Source: US 2026/0368214 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 9 sheets

9 drawing sheets from US 2026/0368214 A1 · click any drawing to enlarge

How apps, databases, and AI share a grading checklist, drawing sheet 1 from US 2026/0368214 A1
Sheet 1 How apps, databases, and AI share a grading checklist
Asking an AI to grade text and suggest rules, drawing sheet 2 from US 2026/0368214 A1
Sheet 2 Asking an AI to grade text and suggest rules
Logging past AI answers to build and update the checklist, drawing sheet 3 from US 2026/0368214 A1
Sheet 3 Logging past AI answers to build and update the checklist
Checking and fixing AI answers live before users see them, drawing sheet 4 from US 2026/0368214 A1
Sheet 4 Checking and fixing AI answers live before users see them
Turning extra AI comments into new checklist questions, drawing sheet 5 from US 2026/0368214 A1
Sheet 5 Turning extra AI comments into new checklist questions
Combining duplicate feedback and trimming unneeded questions, drawing sheet 6 from US 2026/0368214 A1
Sheet 6 Combining duplicate feedback and trimming unneeded questions
Rewriting a draft answer if the quality scores low, drawing sheet 7 from US 2026/0368214 A1
Sheet 7 Rewriting a draft answer if the quality scores low
An office app assistant suggesting formulas using vetted responses, drawing sheet 8 from US 2026/0368214 A1
Sheet 8 An office app assistant suggesting formulas using vetted responses
Show the other 1 sheets
Hardware parts that run the evaluation software, drawing sheet 9 from US 2026/0368214 A1
Sheet 9 Hardware parts that run the evaluation software
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.