Google Files Patent for an AI That Turns Plain-English Orders Into Robot Programs
Google's patent has an AI write a short computer program every time you give a robot an order, then runs that program on the robot. The AI works from a menu of what the robot can see and do.
What Google's robot-code patent actually does
You're standing next to a robot arm. There are a few blocks and an empty bowl on the table. You say, 'Stack the blocks on the empty bowl,' and the robot just does it, even though nobody programmed that exact task ahead of time.
Here's the trick Google describes. Instead of training the robot on thousands of examples of each task, an AI model that learned from huge amounts of computer code writes a short program on the spot. The program uses abilities the robot already has, like 'spot an object with the camera' and 'move the gripper there,' and strings them together in a new order. Then the robot runs it.
If the AI writes code that leans on a step that doesn't exist yet, the system asks it to write that missing piece too. The same setup can also answer questions about what the robot sees.
obtaining a natural language instruction for interacting with the robot; generating an input sequence that comprises: (i) the natural language instruction, and (ii) a context sequence that specifies one or more application programming interfaces (APIs) that can be called to obtain outputs generated from one or more sensors of the robot, to control the robot, or both; …
Translation: The system takes a normal English command and pairs it with a list of available robot controls and sensors.
How the AI writes and fills in robot programs
The system starts with your instruction (typed or spoken) plus a context sequence: a menu of the APIs the robot offers. An API (application programming interface) is a named command that software can call. Some read the robot's sensors, like a camera-based object finder. Others control it, like setting its speed.
Both go into a code generation neural network, an AI model trained to predict the next piece of computer code, the way a chatbot predicts the next word. It writes a program, in a language such as Python, that calls those APIs. Shown an example like 'if you see an orange, move backwards,' it turns 'move rightwards until you see the apple' into a loop that keeps moving until the apple appears.
Then comes hierarchical program generation:
- The system scans the code for functions it calls but nobody defined.
- For each gap, it asks the model again to write that function.
- It repeats until every function exists, then runs the program.
At run time, a global dictionary (a lookup table) lists the available APIs. A local dictionary starts empty and collects values the program creates, which can come back to you as an answer. The filing also describes a safety check before the program runs.
A language model program is computer program generated from an output of a code generation neural network, e.g., one that has been trained on a language modeling objective on computer code data.
Translation: An AI trained on computer code automatically writes the software used to run the machine.
What this means for robots that follow orders
If this works the way the filing describes, you could give a robot a new job just by describing it, with no engineers programming or retraining it first. The text covers robot arms, humanoids, four-legged robots and even self-driving vehicles, and it says the same setup can answer questions about the robot's surroundings.
For now this is a patent application with no product attached. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. The filing claims priority to an application from September 2022, so the idea has been around for a while.
Google's 55th filing we've tracked since May on AI agents acting for you follows work on robots checking their own limits and one button finishing chores across apps.
The big cost here is trust. A robot running code an AI wrote seconds ago is only as safe as that code, and the filing's main guard is a quick scan that rejects things like import statements and a couple of risky commands. That is a thin fence around a machine that moves in the physical world.
The second cost is a ceiling. The AI can only mix the actions on the menu it is handed, so a robot with few skills stays limited however well it follows orders. Each missing step also needs another round of asking the model.
The trade still reads as worth it, since the filing says the alternative is piles of expensive training data from real robots. No test results appear in the text, though, so the real answer is still open.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0316268 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in