Microsoft Files Patent for Training AI on Whole Sets of Graded Answers
AI chatbots can learn to be brief for a bad reason: the example answers they train on are often short. Microsoft's patent application describes a training method built to push back on that habit.
What Microsoft's answer-grading patent does
Every time you ask a chatbot a question, it has to decide when to stop talking. Too soon, and you get a thin answer that skips the part you needed. According to Microsoft's filing, that habit often starts in training, because the example answers written by people tend to be short.
The patent describes a way to train a language model using many graded answers to the same question at once. Each answer gets a score. Answers scoring above average are nudged up, answers below average are nudged down, and the ones furthest from average count the most.
The filing also describes a safeguard that penalizes the model for ending an answer too early, so it can't win just by going short. And the method skips a second 'reference' copy of the model, which the company says saves memory during training.
… partitioning the associated responses into a first set containing responses with associated rewards that are above the mean reward and a second set containing responses with associated rewards that are below the mean reward …
Translation: It splits the answers into two piles depending on whether they scored above or below the average reward.
How the training splits good answers from bad
The filing calls its method REFA, short for reference-free alignment. It starts with a dataset of questions, each paired with several candidate answers, and every answer carries a reward value, a number for how good it is (the filing says a separate scoring model can supply these).
For each question, the system averages the rewards and splits the answers into two sets: above average and below average. It then gives each answer a quality score, a measure of how certain the model was about each word (token) it wrote, averaged over the answer's length so longer answers get no penalty for having more words. Each answer also gets a weight based on how far its reward sits from the average. One example uses an exponential formula, so standouts and flops count extra.
The weighted scores are summed into one total per set (called partition functions), and the two totals feed a loss function, a number that tracks how far the model is from the behavior wanted. The losses are averaged across the dataset, and the model's parameters are updated by gradient descent (many small adjustments that push that number down).
Dependent claims add extras: an end-of-sequence regularizer that checks how likely the model is to produce an 'end' token where real answers actually finish, and penalizes stopping early, plus settings for sharpness and for how hard to punish below-average answers. 'Reference-free' means no second, frozen copy of the model is needed for comparison.
… mitigating biases toward brevity, resulting in richer and more detailed outputs.
Translation: The system improves AI training while preventing the model from giving overly short answers.
Why chatbot answer length is a training problem
If you use an AI assistant for customer support chats, homework help or health questions, the length and depth of its answers matters. The filing's stated aim is richer, more detailed replies, since people asking about hard topics tend to want fuller explanations than the short examples a model may have learned from.
There is a cost angle too. The filing says dropping the second reference model cuts memory needs during training and speeds things up, which matters when models are huge. It is a training recipe, so you would never see it directly, only possibly in how a model answers. The filing describes the method and does not name any product.
Microsoft's 64th filing in the AI assistant and agent work we've tracked since May joins earlier applications like one giving game characters memory and agents behind one chat window.
Claim 1 is long and specific. It covers a training recipe with every step spelled out: average the scores for a question's answers, split them into above and below average, measure how confident the model was in each answer word by word, weight each answer by its distance from the average, and add it all up into one number the model is adjusted to shrink.
Notice what claim 1 leaves out. The answer-length safeguard that the abstract plays up appears only in claim 2 and later claims, so the broadest version covers the scoring and weighting recipe alone. Its steps are narrow and its use is wide, since the wording says any language model trained this way, with no tie to a product or task.
If granted, the claim would reach training setups that follow this full sequence. A team that scored or weighted answers a different way would be working outside this wording. My read: a specific piece of AI-training math, with a pitch about longer answers that the first claim does not actually require.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0310954 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in