Get the newsletter
Microsoft · AI / ML · Filed Jun 26, 2025 · USPTO published Nov 19, 2026 · This week's release

Microsoft Files Patent for a Safety Test That Chatbots Can't Study For

Chatbot makers can study for a safety test the way students study for an exam, if the questions leak. Microsoft's patent application describes an exam that gets rebuilt on a schedule, so there is no answer key to memorize.

An evaluation system creates tricky test questions across different types of harm to score how well competing AI models resist abuse. Drawing from patent filing US 2026/0368060 A1.
An evaluation system creates tricky test questions across different types of harm to score how well competing AI models resist abuse.All 4 drawings ↓
Publication number
US 2026/0368060 A1(US20260368060A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Jun 26, 2025 · Nov 19, 2026
Inventors
Ahmed Elgohary GHONEIM, Amin SAIED, Kyle Philip JACKSON
Status when we published
Waiting for an examiner
Document
20 claims
US classification
702/186
Parent application
Claims priority from a provisional application 63806172 (filed May 15, 2025)

What Microsoft's chatbot safety exam does

Every time a new AI chatbot launches, its maker says it's safe. And every time, you're left taking their word for it, because there's no agreed way to compare one chatbot against another.

Microsoft's patent application describes a safety exam for chatbots. The exam is a list of trick questions, like the ones a bad actor might use to coax out weapon instructions, scam scripts or hate speech. Each chatbot is graded on how much harmful material it hands over. Every model gets the same exam during a set stretch, such as a week or a month, so the scores can be compared fairly.

Here's the twist: the exam doesn't stay put. When the stretch ends, a fresh version is built from a big pool of tricks, and any question too close to an old one is thrown out. That keeps a chatbot maker from studying the answer key.

From the filing · CLAIM 1
populating a sampling pool with a set of jailbreak attempts engineered to exfiltrate harmful content from a language model, the set of jailbreak attempts pertaining to multiple seed categories identifying different types of harm …

Translation: The system builds a test bank of tricky prompts designed to trick AI into generating dangerous material across various topics.

How the exam picks tricks from its pool

The filing describes two pieces of software: one builds the test ahead of time, the other gives it to chatbots and scores them. The builder asks a language model (the AI behind chatbots) to write jailbreak attempts, meaning prompts built to talk a model past its safety rules. It does this for each seed category, a type of harm such as physical or data privacy harm, using several preset construction methods.

Every attempt is fired at a group of test models. Its success score is the share of those models it fooled, so 90% means it worked on 90% of them. Attempts under a cutoff (the filing's example is 20%) are dropped, and the rest go into a sampling pool.

To assemble one version of the test, the system:

  • Removes attempts that a language model judges too similar to earlier tests;
  • Picks an equal number from each harm category (for example, two from each of ten categories for a 20-question test);
  • Picks evenly across success scores, so the test mixes easy and hard tricks and can tell models apart.

Claim 1 itself only requires the even split by harm category. In live testing, a scoring language model rates each reply from 1 to 10 on how closely it matches the harmful goal, and the results appear on a display.

Why comparable chatbot safety scores matter

If you're choosing an AI assistant for work, school or your kids, you mostly have each company's word that its product is safe. A shared exam with scores for each type of harm would give you something to compare. The filing points at the pressure behind this: governments and regulators are asking AI makers for more transparency and accountability.

The filing also says model designers struggle to know how their products measure up against other providers' models. A shared yardstick that resets on a schedule would let a company see where it stands during the same week or month as its rivals. This is a patent application, so nothing here says when, or whether, such scores would ever be published.

This is the 35th Microsoft filing we've tracked since July in our AI guardrails race, following one on keeping AI out of certain folders and one on a self-rewriting grading checklist.

Source: US 2026/0368060 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 4 sheets

4 drawing sheets from US 2026/0368060 A1 · click any drawing to enlarge

How test prompts are generated to grade AI safety, drawing sheet 1 from US 2026/0368060 A1
Sheet 1 How test prompts are generated to grade AI safety
Organizing trick questions by harm category and difficulty, drawing sheet 2 from US 2026/0368060 A1
Sheet 2 Organizing trick questions by harm category and difficulty
Steps for scoring chatbots against harmful test prompts, drawing sheet 3 from US 2026/0368060 A1
Sheet 3 Steps for scoring chatbots against harmful test prompts
Computer hardware that runs the safety evaluation system, drawing sheet 4 from US 2026/0368060 A1
Sheet 4 Computer hardware that runs the safety evaluation system
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.