Waymo Files Patent for Freeze-Frame Replays of Its Self-Driving Car's Decisions
Waymo's patent describes a way to freeze one moment of a self-driving car's decision-making and replay it later. The replay takes tens of seconds, where a full simulation can take minutes or hours.
What Waymo's freeze-frame testing actually does
You're riding in a self-driving car when it brakes hard for a cyclist you never noticed. Back at the office, an engineer wants to know exactly why it made that call, and what would have happened if the cyclist had been a little farther away.
Waymo's patent describes a way to freeze that moment. When something interesting happens, or every few seconds, the car saves a small bundle of exactly what one piece of its driving software was looking at, and nothing more. Later, engineers can reopen the bundle and rerun just that piece in tens of seconds, instead of the minutes or hours a full simulation can take.
Because the moment stays frozen, they can also nudge the details, like moving a pedestrian from two meters away to one, and watch how the software reacts.
… the set of state information being a minimal amount of information needed to is associated with recreating an action or the state of the component at the particular point in time; …
Translation: It captures only the bare minimum data required to replay a specific moment.
How Waymo saves and replays a single driving moment
The patent covers a way to test one piece of a self-driving system at one instant. The piece might be the planner (which decides where the car goes over the next few seconds), the behavior prediction module (which guesses what other road users will do) or the perception module (which makes sense of sensor data).
The claims describe these steps:
- Save the minimal information needed to recreate what the component did, and serialize it (pack it into a compact, storable format; the filing names protocol buffers as one option).
- Later, deserialize it (unpack it back into working form) and hand it over for evaluation.
- Rebuild the component's internal state to match the original, then run it again.
A snapshot can be triggered by a flag, a timer, or a request from another module or a remote service. The description gives examples: saving one every 2 to 5 seconds, or whenever the planner predicts a collision or a component runs too long. It works on real drives and on simulations.
Other claims cover a safety check and a tweak feature. The check compares the unpacked data to the original and flags any mismatch. The tweak step alters items in a snapshot, repeats that to build a whole set, and defines metrics from the results. The description's example varies a car's speed and the type of object in the next lane.
Each snapshot contains a serialized set of inputs necessary to evaluate the particular component. These inputs comprise the minimal amount of information needed to accurately and faithfully recreate what the component did or does.
Translation: The system saves just enough input data to accurately replay past vehicle decisions.
Why quicker testing matters for self-driving fleets
You would never see this feature, but you could feel its effects. A car that makes a strange call is only fixed once engineers can recreate exactly what it saw. Snapshots give them a cheap, repeatable way to do that, including for rare scenes such as a pack of cyclists on a residential street or a car backing up along a freeway.
The filing also says many parts of a driving system have no clear pass or fail test at a single instant. By collecting lots of snapshots, engineers can define measures instead, such as the predicted distance to other objects over the next second. That makes it easier to tell whether a software change made a tricky situation better or worse before it reaches the road.
Google's 79th filing we've tracked in our Waymo robotaxi sensing watchlist since May builds on work like spotting a falling pedestrian and avoiding double left-turn blind spots.
Nothing here needs new hardware. It is software that runs on the computers a Waymo car already carries and on the servers used for testing. The shortest route to use is the one the filing lays out: a flag turns on, snapshots land in a database, and engineers rerun them.
That also means you will never see it as a feature. It shows up as better testing, and later as software that handles odd situations with fewer surprises, if it works. The filing continues a 2021 application, so the idea has been around for a while.
It is sensible internal tooling. It is close to shippable inside the company and nowhere near a product, which is exactly where an engineering tool should be.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0346855 A1 · click any drawing to enlarge
Show the other 2 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in