IBM Files Patent to Train Chatbots From Separate Piles of Good and Bad Answers
IBM has filed a patent application for teaching a chatbot right from wrong using two loose piles of rated answers, with no matched good-and-bad pair for each question.
What IBM's good-pile, bad-pile training does
Imagine you run a help chatbot for a hospital. Your staff has flagged some of its answers as good and others as bad, but those answers respond to different questions. You never got a tidy pair of one good reply and one bad reply to the same question.
IBM's patent application describes a way to teach the chatbot anyway. Instead of judging each answer on its own, the system scores the whole pile of good answers and the whole pile of bad answers. Then it nudges the chatbot until the good pile scores higher than the bad pile across the board.
In practice, it lines up both piles from lowest score to highest. The weakest good answer should beat the weakest bad one, the middle should beat the middle, and the best should beat the best. Wherever that fails, the chatbot gets corrected.
… determining by a reward component, based on the current response, a reference response and preference data, a reward distribution of positive responses and a reward distribution of negative responses …
Translation: The system sorts chatbot answers into separate groups of good and bad responses based on how well they match expectations.
How IBM lines up the best and worst answers
When a prompt arrives, the model writes a current response. A reward component (a scoring step that rates how desirable an answer is) compares it with a reference response, which is what the same model said before any training. It also uses preference data, meaning answers that people, or another AI, marked as chosen or rejected.
From this, the system builds two reward distributions (a spread of scores): one for positive responses and one for negative ones.
An optimizing instruction then goes to the model's optimization interface. It adjusts the model using an alignment optimal transport solution (a math method for matching two spreads of numbers at the lowest cost). The aim is for positive scores to dominate negative scores at every rank. In plain terms:
- Sort the positive scores from lowest to highest.
- Sort the negative scores the same way.
- Penalize any spot where a positive score sits below the negative score at that rank.
The filing calls this a convex relaxation, which means softening a strict rule into a penalty smooth enough to train against. It works with unpaired data, and with paired data, where the model's good-versus-bad margin is compared against the untrained model's margin. The description adds that the same idea could apply to images and audio, not only text.
… issuing an optimizing instruction to an optimization interface on the machine learning model, execution of the optimizing instruction on the machine learning model causing alignment of the machine learning model with the preference data …
Translation: The model receives instructions to adjust itself so it produces better answers in the future.
Why chatbot makers in regulated fields care
Companies that put chatbots in front of customers in healthcare, law and finance need them to stay inside safe, approved behavior. The filing says good labeled examples are hard to get before launch. A method that works from two separate piles of rated answers, with no matched pairs, could lower the amount of careful labeling work involved.
The description also says the math looks at whole groups of answers instead of single ones, which it claims saves computing effort. For you, the result would be a chatbot that gives fewer harmful or misleading replies, such as bad medical advice. The filing describes a training method, and it names no product.
IBM's 64th filing we've tracked since May in the AI guardrails race adds to a pattern that includes one booting staff from calls and one guiding chatbot source picks.
Claim 1 is wider than the sorting trick that gets the attention. It asks for three things: a chatbot that answers a prompt, a scoring step that builds one spread of scores for good answers and one for bad answers (using the model's pre-training answer as a baseline), and a tuning step that uses an optimal transport solution to line the model up with preferences.
Notice what claim 1 leaves out. Sorting, unpaired data, and the rule that good scores beat bad scores at every rank all appear only in the later claims. If granted, the core claim would reach any training run that fits those three steps, however the math is arranged inside.
The optimal transport wording sets the boundary. The description says an existing method called direct preference optimization scores answers one at a time, which makes it a different approach from the one claim 1 names.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0330291 A1 · click any drawing to enlarge
Show the other 4 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in