Waymo Patents a System That Predicts All Nearby Drivers' Moves at Once
Most prediction systems think about one driver at a time. Waymo's new patent describes a way to predict what every car, cyclist, and pedestrian in a scene will do next, all together, simultaneously, which changes how a self-driving car thinks about risk.
How Waymo reads a crowd of drivers in one shot
Today's self-driving cars face a hard problem: the road is full of people making decisions that affect each other. If one driver brakes, another might swerve. Predicting those moves one person at a time misses that connection entirely.
Waymo's patent describes a system that looks at the whole scene, every nearby driver, cyclist, and pedestrian at once, and generates a single, coordinated prediction of what everyone will likely do next. Instead of running separate guesses per person, it produces a joint forecast that accounts for how each person's choices ripple into everyone else's.
To do this, the system converts movement into small, discrete chunks called motion tokens, like letters in a word, and then strings them together into a predicted path for each person simultaneously. The result is a coherent picture of where the whole crowd is headed, not just a pile of individual guesses.
… generating a respective predicted future trajectory for each of a plurality of agents in the scene at the current time point by sampling a sequence of discrete motion tokens that defines a joint future trajectory for the plurality of agents …
Translation: The system guesses where all nearby people and cars will move at the same time.
How the neural network samples joint motion tokens
The core of the patent is a trajectory prediction neural network that takes in scene context data (sensor readings, map information, and the current positions and velocities of all agents nearby) and outputs predicted future paths for every detected agent at the same time.
The key mechanism is discrete motion tokens. Rather than predicting a smooth curve directly, the system encodes movement into a vocabulary of small, fixed motion units, similar to how autocomplete predicts the next word in a sentence. It then samples a sequence of these tokens for each agent, building up a predicted path step by step.
Crucially, the sampling is joint: the network generates tokens for all agents together, so the predicted path for the car on the left is aware of, and consistent with, the predicted path for the pedestrian crossing ahead. This is different from running independent predictions and hoping they don't contradict each other.
By conditioning the entire process on shared scene data from a single time point, the system can produce multiple plausible futures (by sampling different token sequences) rather than one single deterministic guess, which gives downstream planning components a richer picture of uncertainty.
What this means for self-driving car safety decisions
For a self-driving car, a prediction that misses how two nearby drivers interact can be dangerous. If your car thinks the truck will go straight while the car next to it cuts left, the safest path looks very different than if it understood those two moves were linked. A joint prediction system closes that gap.
For you as a passenger, this kind of improvement is invisible but consequential. It means the car should make fewer overcautious stops and fewer missed-hazard calls, because its mental model of the road is closer to how things actually unfold. Waymo has been filing around scene-level prediction since at least 2024, and this patent fits a pattern of treating the driving environment as a single interacting system rather than a collection of independent objects.
Google's 58th filing we've tracked in the self-driving sensing race since May builds on earlier applications like one on steering around icy roads and one on planning gear shifts early.
Converting a car's smooth, continuous movement into fixed chunks sacrifices precision by design. Every prediction starts slightly wrong because the real world does not move in neat steps, and how wrong depends on how finely those chunks are defined, which this document does not specify.
Predicting twenty cars at once, with each influencing the others at every moment, multiplies the processing load in ways that could strain the tight time budgets that safe driving demands. That is a real cost, and it scales with traffic density, meaning the hardest situations are also the most expensive to compute.
The trade still reads as worth it. A system that treats a busy intersection as one interacting scene is more honest about how traffic actually works than one that treats each car as a stranger to the others, and for a company whose entire business depends on anticipating what happens next, that coherence is the point.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0285374 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in