Get the newsletter
Google · AI / ML · Filed Aug 11, 2025 · USPTO published Nov 19, 2026 · This week's release

Google Patents a Robot Planner That Checks the Room Before Each Step

Tell a robot to wipe the table, and it may charge ahead even if a soda can is sitting in the way. Google's patent has the robot look first, rewrite its plan, and ask another robot or a person for help.

A mobile helper robot receives a spoken instruction from a person to prepare fruit in a kitchen. Drawing from patent filing US 2026/0364803 A1.
A mobile helper robot receives a spoken instruction from a person to prepare fruit in a kitchen.All 7 drawings ↓
Publication number
US 2026/0364803 A1(US20260364803A1)
Applicant
GOOGLE LLC
Filed · Published
Aug 11, 2025 · Nov 19, 2026
Inventors
Nikhil J. Joshi, Anthony G. Francis, Jr.
Status when we published
Approved; patent expected soon
Document
17 claims
US classification
700/246
Examiner
CAIN, AARON G (Art Unit 3656)
Parent application
National Stage Entry of PCT/US2024/018110 (filed Mar 1, 2024)(PCTUS2024018110)

What Google's look-then-replan robot actually does

Every time you ask someone to do a chore, they glance around and adjust. "Wipe the table" turns into "move the cup first, then wipe." Most robots can't do that. They follow a list of steps and assume the world matches it.

Google's patent describes a way to give a robot that glance. A language AI (the kind behind chatbots) turns your plain-English request into a rough to-do list. Then the robot takes a picture and a vision AI answers a question about it, such as "is there fruit on the table?" The answer goes back to the language AI, which fixes the list. "Pick up the fruit" becomes "pick up the pear."

If the robot is stuck, say it has a wiping tool but needs to grab something, the plan can include asking another robot or a person for help. Then it carries on with the original job.

From the filing · CLAIM 1
… processing, using the VLM, the one or more instances of vision data and the VLM prompt to generate a task conditioned description of the environment; processing, using the generative model, the instruction and the task conditioned description of the environment to generate refined natural language text output that indicates one or more refined sub-tasks for performing the task …

Translation: The robot looks at its surroundings using a camera and updates its plan before taking the next step.

How the robot rewrites its plan after each look

The patent covers a loop that combines two kinds of AI. First, a generative model (usually a large language model, or LLM, the same family as chatbots) takes a free-form instruction like "get the fruit ready to wash" and writes out sub-tasks, such as "navigate to the table" and "pick up the fruit."

Next, the system writes a question for a visual language model (VLM, an AI that answers questions about images) based on those sub-tasks. In the filing's example the question is "is there fruit on the table?" The VLM looks at the robot's camera data and returns a task conditioned description, meaning a summary of the scene focused on the job at hand. Here it says there is a pear.

The LLM then reads the original instruction plus that description and writes a refined plan. The robot compares the refined plan to written descriptions of the skills it has (for example, "pick up the object") and runs the matching one.

  • After each skill, the robot looks again and checks whether it worked.
  • A failure changes the next step. The filing describes sending a message to another robot that has the right tool.
  • A robot can also check itself, for example whether it has a grasping tool at all.

The loop repeats until the task is done, time runs out, or no skills remain.

From the filing · THE ABSTRACT
Some implementations relate to utilizing both a large language model (LLM) and a visual language model (VLM) in generating, and at least selectively refining, a plan for the execution of a long-horizon robotic task.

Translation: The system uses two different AI models together to figure out and adjust long tasks.

Why a robot that asks for help matters at home

If you ever have a home or office robot, the difference between a toy and a helper is what happens when something goes wrong. A robot that sees the clutter, works out what it needs, and calls for backup is far more useful than one that wipes around the soda can or freezes. The filing also describes robots with different tools teaming up, so one with a wiper can recruit one with a gripper.

The application has been approved by the patent office. Once Google pays the final fee, it should be issued as a granted patent. Google's interest in robots that follow plain-English instructions shows up in the background material, which cites earlier Google planning work for robots.

One more thing to know: the patent office has already approved this application. Once Google pays the final fee, it should be issued as a granted patent, and the final claims may read a little differently from the version described here.

This is the 65th Google filing we've tracked since May on AI agents working for you, joining a video-call AI chat and voice-command earbuds.

Source: US 2026/0364803 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 7 sheets

7 drawing sheets from US 2026/0364803 A1 · click any drawing to enlarge

A person giving a spoken command to a household robot, drawing sheet 1 from US 2026/0364803 A1
Sheet 1 A person giving a spoken command to a household robot
Looking at kitchen objects from above to plan movements, drawing sheet 2 from US 2026/0364803 A1
Sheet 2 Looking at kitchen objects from above to plan movements
Combining spoken instructions with camera data to choose actions, drawing sheet 3 from US 2026/0364803 A1
Sheet 3 Combining spoken instructions with camera data to choose actions
Checking room changes after each step to plan next moves, drawing sheet 4 from US 2026/0364803 A1
Sheet 4 Checking room changes after each step to plan next moves
Using vision data to pick the right command skill, drawing sheet 5 from US 2026/0364803 A1
Sheet 5 Using vision data to pick the right command skill
Hardware parts that let the robot sense and manipulate objects, drawing sheet 6 from US 2026/0364803 A1
Sheet 6 Hardware parts that let the robot sense and manipulate objects
Computer parts that process language and vision commands, drawing sheet 7 from US 2026/0364803 A1
Sheet 7 Computer parts that process language and vision commands
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.