Get the newsletter
IBM · AI / ML · Filed May 16, 2025 · USPTO published Nov 19, 2026 · This week's release

IBM Files Patent for Testing Chatbots With Angry, Sloppy, and Overly Formal Rewrites

Chatbots can answer a polite question correctly and flub the same question typed in anger. IBM's patent application describes a way to manufacture those angry, sloppy and overly formal versions on purpose, so the weak spots show up in testing instead of in front of customers.

Computers and cloud servers run software that rewords sentences to test how well chatbots handle sloppy or emotional messages. Drawing from patent filing US 2026/0368332 A1.
Computers and cloud servers run software that rewords sentences to test how well chatbots handle sloppy or emotional messages.All 5 drawings ↓
Publication number
US 2026/0368332 A1(US20260368332A1)
Applicant
INTERNATIONAL BUSINESS MACHINES CORPORATION
Filed · Published
May 16, 2025 · Nov 19, 2026
Inventors
Mikael Hillborg, Gabriela Rodriguez-Florido, Alex Wilson, Aurangazeeb A K
Status when we published
Examiner raised objections; the company can respond
Document
20 claims
US classification
726/25
Examiner
NGUY, CHI D (Art Unit 2435)

What IBM's chatbot stress-test patent does

You're typing to a company's support chatbot, you're annoyed, and you rush the message, typos and all. The bot gives you an answer that's off, even though a calm, polite version of the same question would have worked fine.

IBM's patent application describes a way to catch that before customers do. Start with one ordinary message. A text-writing AI rewrites it over and over, each time using a different random mix of styles: more formal, more casual, angrier, sloppier, wordier or vaguer. A check then throws out any rewrite that no longer means the same thing as the original.

What's left is a pile of messages that all say the same thing in very different voices. Feed them to the chatbot, and any that trip it up show you where it's weak.

From the filing · CLAIM 1
generating a plurality of rewritten texts from an original text using an AI text generator configured for natural language processing and generation, wherein generating each rewritten text comprises using the AI text generator to apply a statistically unique sequence of operations to the original text …

Translation: The system creates many variations of a base sentence by running a distinct series of text changes using another AI.

How IBM scrambles a question and checks meaning

The method starts with an original text, such as a customer question. For each rewrite, it builds a sequence of operations, which are instructions to an AI text generator like 'make it formal,' 'make it angry,' or 'add misspellings.' The operations can be picked by a user, pulled from a stored list, or generated on the spot from a description of what the text is about.

Each operation gets a random strength. Those strengths are normalized (scaled so the biggest equals 1) and then multiplied by a complexity level, which controls how much of the text may change. The randomness makes every sequence statistically unique: a very high chance, though not a guarantee, that no two rewrites match. Different batches can use different complexity levels.

Next comes the filter. A distance function (a calculation that compares two texts, for example by comparing numeric fingerprints of their meaning called vector embeddings) checks whether the rewrite stays within a set threshold of the original. Rewrites that pass go into the challenge set; the rest are dropped. The loop repeats until the set reaches a size the user chose.

Users can also mark key words or phrases (called entities) to leave alone, or to be the only parts rewritten. The description adds that a clustering step on the chatbot's responses could reveal which combinations of operations make it fail.

From the filing · THE ABSTRACT
Semantic similarity of each rewritten text to the original text is determined; and each rewritten text that has been determined to be semantically similar to the original text is added to the set of challenge texts.

Translation: It checks if the rewritten sentences still mean the same thing before saving them as test questions.

Why chatbot makers care about odd phrasing

Businesses already use chatbots to answer simple questions, and real customers don't write like test scripts. They type in a hurry, get frustrated, and misspell things. A tool that produces those variations automatically could help a company find weak spots before you run into them. The filing says the approach is computationally efficient (cheap to run) because the similarity check is a simple calculation.

For any company building or running an AI system, showing that it behaves the same when the wording changes is a practical need. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. Nothing in the filing points to a specific IBM product built on it.

That makes this IBM's 81st filing we've tracked since May in the AI guardrails race, joining one on fake planning problems and one on plain-English alert rules.

Source: US 2026/0368332 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 5 sheets

5 drawing sheets from US 2026/0368332 A1 · click any drawing to enlarge

Hardware and cloud networks that run the chatbot testing software, drawing sheet 1 from US 2026/0368332 A1
Sheet 1 Hardware and cloud networks that run the chatbot testing software
How the system creates varied test sentences with matching meanings, drawing sheet 2 from US 2026/0368332 A1
Sheet 2 How the system creates varied test sentences with matching meanings
Building a customized mix of text-changing rules by difficulty level, drawing sheet 3 from US 2026/0368332 A1
Sheet 3 Building a customized mix of text-changing rules by difficulty level
Components that produce and store rewritten test prompts, drawing sheet 4 from US 2026/0368332 A1
Sheet 4 Components that produce and store rewritten test prompts
Feeding altered prompts to a chatbot to uncover its weaknesses, drawing sheet 5 from US 2026/0368332 A1
Sheet 5 Feeding altered prompts to a chatbot to uncover its weaknesses
Patent filing page

More in AI assistant & agent

All 436 AI assistant & agent patents →

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.