Salesforce Files Patent for Automated Bots That Shut Themselves Off When Accuracy Drops
AI assistants can give a different answer every time you ask the same question, which makes them hard to grade. Salesforce's filing describes a watchdog that scores each answer on its own checklist and switches the assistant off when any single score drops too low.
What Salesforce's AI agent watchdog actually does
Ever asked a chatbot the same question twice and gotten two different answers? Both might be fine, or one might be wrong, and that is a headache for any company letting an AI talk to its customers.
Salesforce's filing describes a watchdog program that reads each answer the AI gives and grades it on separate items, such as how easy the text is to read, how concise it is, and whether details like an order number or date match what they should. Each item has its own passing grade.
If even one score falls below its line, the watchdog can turn the AI agent off. It can also look back at recent changes to the agent and undo them, so the agent returns to the version that worked.
… determining that at least one of the generated metric scores is lower than a threshold metric score for the component metric of the at least one of the generated metric scores, wherein the generated metric scores are compared to threshold metrics scores independently; and disabling the non-deterministic system.
Translation: The bot turns itself off the moment its performance falls below an acceptable level in any single category.
How the evaluator scores and shuts off an agent
The patent covers an evaluator, a piece of software that sits beside an AI agent and checks its work. The filing calls the agent a "non-deterministic system," meaning one that can give different outputs for the same input, usually because it uses a large language model (LLM, the type of AI behind modern chatbots).
The evaluator receives the agent's output and picks component metrics, which are individual yardsticks. Some are natural language metrics (readability, conciseness, completeness, coherence, whether the reply matches the intent of the request). Others are transactional metrics, which check data, such as how many attributes in the output match the expected ones. If the output has no written text, only the data checks apply.
Each yardstick has a calculator that produces its score. Users can swap calculators, for example replacing a default readability formula with Flesch-Kincaid. A user-written file called an adapter can list which metrics and calculators apply to a given agent.
Every score is compared on its own to that metric's threshold, rather than blended into one grade. If any score falls short, the claims cover disabling the agent. Dependent claims add finding recent changes to code or to the LLM's weights (its learned settings), reverting to the earlier state, and turning the agent back on.
Why a kill switch matters for business AI chatbots
If you have ever dealt with a company chatbot that confidently got your order wrong, you can see the appeal. A tool that catches a drop in quality quickly, and pulls the agent offline before more customers see bad answers, is something businesses running AI agents would want. Checking frequently also helps tie a problem to the specific change that caused it.
The filing also shows why scoring separate items matters: a single overall grade can hide why an agent is failing. The description says a lower score on one item, such as readability, points straight at the problem. Salesforce lets customers choose their own metrics and cutoffs, which fits a company selling AI agents to many different businesses.
Salesforce's 19th filing we've tracked since May in our AI guardrails race watchlist extends earlier work on auto-killing bad results and grading meaning vs. wording
Getting from this patent to a working product requires no new hardware and no exotic research, only software that reads an agent's replies, runs some scoring routines, and flips a switch. The scoring methods it describes, such as checking readability or matching facts against a database, are already standard tools in any software team's kit.
The hard decision is where to draw the lines. Set the cutoff too strict and the system shuts down a perfectly fine agent over a minor stumble; set it too loose and bad answers sail through unchecked. The filing hands that judgment to the customer, which is honest but also puts the hardest call squarely on them.
The shortest route to a product is a monitoring dashboard bundled with tools businesses already use to run automated agents, with automatic shutdown available as an option rather than a default, since most companies would want a human alert before their customer support bot goes dark.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Salesforce patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0329971 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in