Google Patents a Robot Planner That Checks the Room Before Each Step
Tell a robot to wipe the table, and it may charge ahead even if a soda can is sitting in the way. Google's patent has the robot look first, rewrite its plan, and ask another robot or a person for help.

- Publication number
- US 2026/0364803 A1(US20260364803A1)
- Applicant
- GOOGLE LLC
- Filed · Published
- Aug 11, 2025 · Nov 19, 2026
- Inventors
- Nikhil J. Joshi, Anthony G. Francis, Jr.
- Status when we published
- Approved; patent expected soon
- Document
- 17 claims
- US classification
- 700/246
- Examiner
- CAIN, AARON G (Art Unit 3656)
- Parent application
- National Stage Entry of PCT/US2024/018110 (filed Mar 1, 2024)(PCTUS2024018110)
What Google's look-then-replan robot actually does
Every time you ask someone to do a chore, they glance around and adjust. "Wipe the table" turns into "move the cup first, then wipe." Most robots can't do that. They follow a list of steps and assume the world matches it.
Google's patent describes a way to give a robot that glance. A language AI (the kind behind chatbots) turns your plain-English request into a rough to-do list. Then the robot takes a picture and a vision AI answers a question about it, such as "is there fruit on the table?" The answer goes back to the language AI, which fixes the list. "Pick up the fruit" becomes "pick up the pear."
If the robot is stuck, say it has a wiping tool but needs to grab something, the plan can include asking another robot or a person for help. Then it carries on with the original job.
… processing, using the VLM, the one or more instances of vision data and the VLM prompt to generate a task conditioned description of the environment; processing, using the generative model, the instruction and the task conditioned description of the environment to generate refined natural language text output that indicates one or more refined sub-tasks for performing the task …
Translation: The robot looks at its surroundings using a camera and updates its plan before taking the next step.
How the robot rewrites its plan after each look
The patent covers a loop that combines two kinds of AI. First, a generative model (usually a large language model, or LLM, the same family as chatbots) takes a free-form instruction like "get the fruit ready to wash" and writes out sub-tasks, such as "navigate to the table" and "pick up the fruit."
Next, the system writes a question for a visual language model (VLM, an AI that answers questions about images) based on those sub-tasks. In the filing's example the question is "is there fruit on the table?" The VLM looks at the robot's camera data and returns a task conditioned description, meaning a summary of the scene focused on the job at hand. Here it says there is a pear.
The LLM then reads the original instruction plus that description and writes a refined plan. The robot compares the refined plan to written descriptions of the skills it has (for example, "pick up the object") and runs the matching one.
- After each skill, the robot looks again and checks whether it worked.
- A failure changes the next step. The filing describes sending a message to another robot that has the right tool.
- A robot can also check itself, for example whether it has a grasping tool at all.
The loop repeats until the task is done, time runs out, or no skills remain.
Some implementations relate to utilizing both a large language model (LLM) and a visual language model (VLM) in generating, and at least selectively refining, a plan for the execution of a long-horizon robotic task.
Translation: The system uses two different AI models together to figure out and adjust long tasks.
Why a robot that asks for help matters at home
If you ever have a home or office robot, the difference between a toy and a helper is what happens when something goes wrong. A robot that sees the clutter, works out what it needs, and calls for backup is far more useful than one that wipes around the soda can or freezes. The filing also describes robots with different tools teaming up, so one with a wiper can recruit one with a gripper.
The application has been approved by the patent office. Once Google pays the final fee, it should be issued as a granted patent. Google's interest in robots that follow plain-English instructions shows up in the background material, which cites earlier Google planning work for robots.
One more thing to know: the patent office has already approved this application. Once Google pays the final fee, it should be issued as a granted patent, and the final claims may read a little differently from the version described here.
This is the 65th Google filing we've tracked since May on AI agents working for you, joining a video-call AI chat and voice-command earbuds.
Source: US 2026/0364803 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 7 sheets
7 drawing sheets from US 2026/0364803 A1 · click any drawing to enlarge

No comments yet