Microsoft Patents a System to Keep Its Chatbot Behaving Consistently Over Time
Chatbots get updated all the time, and each update can change how they sound. Microsoft's patent application describes a way to catch that by staging practice conversations between two AIs and grading both on traits like friendliness and profanity.
What Microsoft's chatbot personality check does
Every time a company updates its chatbot, the bot's personality can shift a little. The short, polite helper you liked last month might turn chatty, or a bit rude, and nobody at the company notices until people complain.
Microsoft's patent describes a way to catch that early. One AI plays the part of a user and chats with the chatbot being tested, over and over. A separate scoring program then reads both sides of every conversation and rates traits like friendliness, formality and profanity, much like a teacher using a grading rubric.
If the scores slip, the system can respond. It might add a nudge to the chatbot's instructions, such as be warmer, or produce a report explaining what went wrong. You would rarely see any of this. You would just get a bot that stays the same kind of bot.
instantiate a plurality of synthetic interaction sessions between a first generative model and a second generative model …
Translation: The system sets up multiple simulated conversations between two different AI models.
How two chatbots grade each other's manners
The core of the filing is an automated test setup. A second generative model (an AI that writes text) plays the user and writes messages. Those messages go into a prompt (the instructions and question handed to the chatbot), and the first model, the one being checked, writes a reply. Many of these practice conversations run automatically.
Every message from both sides goes to a rubric classifier, a scoring program trained on example conversations that humans or machines had already rated, typically on a 1 to 5 scale. The traits can include friendliness, formality, empathy, profanity, verbosity, tone, factuality and creativity. The filing says the scorer can be small and light, with a separate one for each trait. Because the simulated user is scored too, the second AI is held to the same rubric.
Models get updated now and then, so the scores are collected at a series of points in time. That produces a time-series (a record of scores over time) that shows whether a bot is drifting.
The system can also act on the scores. In one example, a reply scored 4 for friendliness against a target of 5, so the system added instructions to use the person's name and show more empathy. It can also roll back to an earlier model version or write a report explaining the score.
Why a chatbot's mood swings are a real problem
If you use a chatbot at work, in a search box or in a customer service window, you expect it to behave the same way next week as it did today. This patent application describes a way for a company to check that automatically, instead of waiting for frustrated users to report a bot that turned snippy or wordy.
The filing also says the same scoring could help meet regulatory or corporate policy requirements. For a company that puts chatbots in front of many people, a built-in gauge of how the bot sounds is practical housekeeping. The filing names no specific product, and a patent application is only a statement of what the company hopes to protect.
Microsoft's 22nd filing we've tracked since July in our AI safety guardrails watch adds to earlier work like one flagging suspicious prompts and one reviewing medical chats.
The problem is real. Chatbots are updated constantly, and the filing itself points out that a polite bot can drift toward swearing or a brief bot toward rambling. A person reading transcripts cannot keep up with every update.
Having one AI play the user and another do the grading fits that scale, since tests can run all day without volunteers. The weak spot is the grader. Its scores are only as good as the human ratings it learned from, and a trait like wittiness or empathy is hard to pin to a number from 1 to 5.
As quality control, the approach is sensible and sized to the job. It flags shifts in manner early, and the filing mentions dashboards and notifications for developers as the place those flags would land. Most people will never know it exists, which is how this kind of plumbing should work.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0310906 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in