Google Patents a System That Scores Every Robot Decision Before It Acts
Most AI agents treat every possible action as equally worth considering at the start of each step. Google's new patent teaches a Transformer to score actions one piece at a time, building on its own previous choices to arrive at a better final decision.
How Google's AI agents pick their next move
Today's AI agents often have to choose from an enormous number of possible actions at every moment, and evaluating all of them at once is impractical. Google's new patent describes a system that breaks that choice down into smaller, sequential steps so the agent can reason its way to a good decision without getting overwhelmed.
Think of it like ordering at a restaurant. Instead of picking an entire meal from scratch all at once, you first choose a protein, then a side, then a drink, each choice informed by the last. Google's Q-Transformer works the same way: at each moment, the agent picks one part of its action, locks it in, then uses that choice to guide the next part.
The system also keeps a running memory of what the agent has already seen and done, so its decisions account for recent history. That makes it especially useful for tasks where context builds over time, like navigating a room or handling objects in a warehouse.
… processing the input sequence of input tokens using a Transformer neural network to select an action to be performed by the agent in response to the current observation, wherein the action comprises a respective sub-action for each of a plurality of action dimensions …
Translation: The system uses a neural network to break down a complex robot movement into smaller, specific sub-tasks.
How the Transformer scores each action dimension in sequence
The patent describes a Q-Transformer neural network designed to control an agent, which could be a robot, a software process, or any system that takes actions in an environment based on observations.
At every time step, the agent collects a current observation (what it sees or senses right now) and combines it with a history of past observations into a sequence of tokens. A Transformer (the same type of architecture behind large language models) then processes that sequence.
The key innovation is in how the system handles action dimensions. Rather than selecting a full action all at once, it treats each component of an action as a separate dimension and resolves them one by one:
- For each dimension, the Transformer generates a Q value (a numeric score representing how good each candidate sub-action is likely to be in the long run) for every possible sub-action in that dimension.
- The highest-scoring sub-action is selected and appended to the sequence as a token.
- That token then becomes part of the context for scoring the next action dimension.
This sequential scoring approach, rooted in Q-learning (a classic reinforcement learning technique where an agent learns the value of actions through trial and reward), lets the Transformer handle high-dimensional action spaces without needing to evaluate every possible combination simultaneously.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment using a Transformer neural network.
Translation: This patent describes software that allows a robot to navigate and interact with its surroundings.
What this means for real-world AI agents and robotics
The problem this patent attacks is real and expensive. Training AI agents to act reliably in physical or complex simulated environments requires enormous amounts of trial-and-error, and that cost scales badly when every action has multiple independent components. A robot arm moving in three-dimensional space already has several degrees of freedom to coordinate; add gripping force, rotation, and timing, and the number of possible actions per moment explodes quickly. Google's sequential scoring approach chips away at that combinatorial burden.
For anyone watching where Google is pointing its AI research, this filing sits squarely in the robotics and autonomous-agent space that the company has been building toward for years. The same Transformer architecture powering large language models is here being adapted for physical-world decision-making, which is a meaningful architectural convergence. Patentlyze tracks new Big Tech patents across the robotics and AI-agent space, where this kind of reinforcement-learning infrastructure work is appearing at an accelerating pace.
The problem Google is addressing here is one of the harder structural issues in applied reinforcement learning: action spaces in real-world tasks are often so large that evaluating options exhaustively per step is computationally ruinous. Breaking action selection into a sequential, token-by-token scoring process is a principled response to that size problem, and doing it inside a Transformer means the agent can draw on its full observation history at every sub-decision. The patent covers foundational method territory rather than a narrow implementation trick, which suggests Google sees this approach as load-bearing infrastructure for its broader agent work.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0236740 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →