Waymo Patents an AI That Predicts Where Every Nearby Driver and Pedestrian Will Go Next
Waymo is borrowing the same kind of AI that generates images from text descriptions and repurposing it to forecast where every car, cyclist, and pedestrian near its vehicles will go next.
How Waymo guesses what everyone around the car will do
You're crossing an intersection and a Waymo robotaxi is nearby. The car needs to decide whether to hold its position, nudge forward, or brake, and it has to make that call based on what every person around it is likely to do in the next few seconds. Getting that wrong, even once, is the difference between a smooth ride and a collision.
This patent describes a system that uses a type of AI, called a diffusion model, to generate those predictions. You may have heard of diffusion models from image-generation tools: they start with noise and gradually refine it into something coherent. Waymo's approach applies the same idea to motion, starting with a rough guess at a future path and refining it across many small steps until the prediction makes sense given everything the car's sensors picked up.
The practical goal is for the car to predict the likely paths of all the agents around it at once, rather than handling each one in isolation, so its decisions account for how everyone's movements interact.
… generating, by using a diffusion model and based on the encoded representation, a trajectory prediction output that predicts a future trajectory for a target agent after the current time point by repeatedly updating an intermediate representation of the trajectory prediction output across multiple steps in a generation process.
Translation: It uses an AI diffusion model to forecast where a person or vehicle will move next through a step by step refinement process.
How the diffusion model builds a path prediction step by step
The system works in three stages.
- Scene encoding: First, the car's sensors gather what the patent calls "scene context data", positions, speeds, and types of all nearby objects at the current moment. A neural network compresses that raw information into a compact mathematical summary of the scene.
- Diffusion-based prediction: A diffusion model then takes that summary and generates a trajectory prediction. Diffusion models work by starting from a noisy, uncertain state and running many small refinement steps until the output is a clean, plausible result. In image generation, the output is pixels; here, the output is a predicted path through space and time.
- Multi-agent output: The system produces a prediction for each target agent in the scene simultaneously, not one at a time, which helps the model account for the fact that a pedestrian's likely path depends partly on what the car next to them is doing.
The claim centers on that iterative refinement loop: the model repeatedly updates an intermediate representation of the trajectory across multiple steps in the generation process. This is different from older approaches that output a single prediction in one pass. The diffusion approach can, in principle, produce a richer distribution of possible futures rather than just a single best-guess path.
What better predictions mean for riders in a Waymo cab
For someone riding in a Waymo vehicle, the benefit is indirect but real. Better trajectory predictions mean the car makes fewer unnecessary hard stops (because it correctly judged that the cyclist was turning away, not cutting in front) and fewer hesitations at intersections (because it correctly predicted that the oncoming driver was slowing down). The ride feels less jerky and more confident.
Waymo has been filing around multi-agent prediction since at least 2022, and this filing pushes further into probabilistic methods. The practical edge of diffusion-based motion prediction, if it holds up in real-world conditions, is that the model can represent uncertainty more honestly, giving the car's planner a realistic sense of how many futures are actually possible, not just the single most likely one.
Google's 61st filing we've tracked since May in the self-driving sensing race adds to a run that includes a two-sensor weather fix and an AI that predicts future views.
For a passenger, the improvement shows up in the moments that feel most chaotic today: busy crosswalks, crowded intersections, parking lots where cyclists, pedestrians, and cars all arrive at once. Because the system builds one shared picture of everyone moving together rather than reading each person in turn, the car can act on what it sees without falling behind the scene unfolding around it.
That lag matters more than it sounds. A robotaxi that creeps and stops and creeps again in a crowd signals to the person sitting in back that the car is confused, and that feeling is hard to shake once it sets in.
The concrete payoff depends entirely on speed. A prediction that arrives too late to act on does nothing for anyone, so the real test is whether the system can run all of this thinking fast enough that passengers simply notice smoother rides through complicated places, and never have to think about why.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0296504 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in