IBM Files Patent for Testing Chatbots With Angry, Sloppy, and Overly Formal Rewrites
Chatbots can answer a polite question correctly and flub the same question typed in anger. IBM's patent application describes a way to manufacture those angry, sloppy and overly formal versions on purpose, so the weak spots show up in testing instead of in front of customers.

- Publication number
- US 2026/0368332 A1(US20260368332A1)
- Applicant
- INTERNATIONAL BUSINESS MACHINES CORPORATION
- Filed · Published
- May 16, 2025 · Nov 19, 2026
- Inventors
- Mikael Hillborg, Gabriela Rodriguez-Florido, Alex Wilson, Aurangazeeb A K
- Status when we published
- Examiner raised objections; the company can respond
- Document
- 20 claims
- US classification
- 726/25
- Examiner
- NGUY, CHI D (Art Unit 2435)
What IBM's chatbot stress-test patent does
You're typing to a company's support chatbot, you're annoyed, and you rush the message, typos and all. The bot gives you an answer that's off, even though a calm, polite version of the same question would have worked fine.
IBM's patent application describes a way to catch that before customers do. Start with one ordinary message. A text-writing AI rewrites it over and over, each time using a different random mix of styles: more formal, more casual, angrier, sloppier, wordier or vaguer. A check then throws out any rewrite that no longer means the same thing as the original.
What's left is a pile of messages that all say the same thing in very different voices. Feed them to the chatbot, and any that trip it up show you where it's weak.
generating a plurality of rewritten texts from an original text using an AI text generator configured for natural language processing and generation, wherein generating each rewritten text comprises using the AI text generator to apply a statistically unique sequence of operations to the original text …
Translation: The system creates many variations of a base sentence by running a distinct series of text changes using another AI.
How IBM scrambles a question and checks meaning
The method starts with an original text, such as a customer question. For each rewrite, it builds a sequence of operations, which are instructions to an AI text generator like 'make it formal,' 'make it angry,' or 'add misspellings.' The operations can be picked by a user, pulled from a stored list, or generated on the spot from a description of what the text is about.
Each operation gets a random strength. Those strengths are normalized (scaled so the biggest equals 1) and then multiplied by a complexity level, which controls how much of the text may change. The randomness makes every sequence statistically unique: a very high chance, though not a guarantee, that no two rewrites match. Different batches can use different complexity levels.
Next comes the filter. A distance function (a calculation that compares two texts, for example by comparing numeric fingerprints of their meaning called vector embeddings) checks whether the rewrite stays within a set threshold of the original. Rewrites that pass go into the challenge set; the rest are dropped. The loop repeats until the set reaches a size the user chose.
Users can also mark key words or phrases (called entities) to leave alone, or to be the only parts rewritten. The description adds that a clustering step on the chatbot's responses could reveal which combinations of operations make it fail.
Semantic similarity of each rewritten text to the original text is determined; and each rewritten text that has been determined to be semantically similar to the original text is added to the set of challenge texts.
Translation: It checks if the rewritten sentences still mean the same thing before saving them as test questions.
Why chatbot makers care about odd phrasing
Businesses already use chatbots to answer simple questions, and real customers don't write like test scripts. They type in a hurry, get frustrated, and misspell things. A tool that produces those variations automatically could help a company find weak spots before you run into them. The filing says the approach is computationally efficient (cheap to run) because the similarity check is a simple calculation.
For any company building or running an AI system, showing that it behaves the same when the wording changes is a practical need. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. Nothing in the filing points to a specific IBM product built on it.
That makes this IBM's 81st filing we've tracked since May in the AI guardrails race, joining one on fake planning problems and one on plain-English alert rules.
Source: US 2026/0368332 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 5 sheets
5 drawing sheets from US 2026/0368332 A1 · click any drawing to enlarge

No comments yet