Google Patents a Coach That Grades Every Search an AI Assistant Runs
Google has filed a patent for a second AI that grades each move a first AI makes, such as running a search or using a calculator. The grades teach the assistant to reach the right answer in fewer steps.
What Google's AI-coach patent actually does
Every time you ask an AI assistant a tricky question, it has to make a small choice: answer from memory, run a web search, or punch numbers into a calculator. Pick wrong and you get a slow, rambling, or incorrect answer.
Google's patent describes a way to teach the assistant to choose better. A second AI acts like a coach, grading each step the assistant takes and asking whether it actually moved toward answering your question. The assistant then adjusts based on those grades, the way you'd get better at a video game by seeing a score after every move.
The coach itself learns from a bigger, better-informed AI that can look up reliable facts while it grades. That lets Google skip much of the slow work of having people review answers by hand. The filing also describes a penalty for taking too many steps, which nudges the assistant to get to the point.
… action feature specifying an action function call to a tool external to the machine-learned agent model; processing, by the computing system and using a machine-learned reward model, the output and the sequence of preceding states to generate a quality indicator that evaluates whether the action feature advances a goal …
Translation: A secondary AI grades whether the assistant actually used an external tool correctly to achieve the goal.
How the grader scores each search and lookup
The patent covers a training loop for an agent model (an AI, such as a large language model, that works through a task one step at a time). At each step the agent produces an action: a function call to a tool outside the model, like a search engine, a database lookup, a calculator, or a private reasoning step the filing calls a thought. The result is added to the running history, and the agent keeps going until it outputs a finish token (a marker meaning it's done).
A reward model (a second AI that acts as the grader) reads the agent's latest action plus the history so far and produces a quality indicator. That can be a yes/no flag, a number, or a word like satisfactory. It judges whether the action moves toward the user's goal. A training program then uses that score to adjust the agent through reinforcement learning (learning from trial, error and scores).
The filing describes two ways to keep the grader well informed:
- The grader can run its own lookups of reference data while it scores.
- Or it can be trained by a larger supervisor model that sees reference material and picks the better of several answers, so that knowledge gets passed down to the grader.
Actions follow a fixed format with start and end markers, and the trainer can add a penalty for each tool call so the agent takes fewer steps.
… generating feedback signals for training a machine-learned agent model. The example method can include obtaining an output of a machine-learned agent model, the output including a next state feature generated by the machine-learned agent model based on a sequence of preceding states.
Translation: The system collects steps taken by the AI agent to learn how it behaves over time.
Why AI assistants that use tools need a coach
If you use an AI assistant that searches the web or does math for you, how well it picks those tools shapes how fast and how right your answers are. A grader that rewards useful searches and penalizes needless ones could mean quicker, more accurate replies.
The filing also pitches this as a cheaper way to train. A small grader replaces much of the human review, and the description says it can run on the same device as the assistant. That could cut energy use and keep more of your data off the network. It's a training method rather than a product, so nothing here tells you when or where you'd notice it.
Google's 27th filing we've tracked in AI models working together since May adds to a pattern that includes their ad-making AI crew and the two-model search ranker.
Claim 1 is wide. It covers training an AI by having a second AI judge whether each tool call, such as a search, moves the assistant toward its goal, then feeding that judgment back as a training signal. It never says the judge must look anything up, and it never names a particular tool.
The narrower details live in the dependent claims: the reference data the grader can fetch, the exact format of an action, and the penalty for taking too many steps. If granted, the broad version would cover any training setup where a grader model scores an assistant's tool calls, whatever the grader's inner workings.
That breadth gives Google a wide foundation. Whether the claim survives examination is a separate question, and this filing is a continuation of an application first filed in July 2023.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0311065 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in