Zoox Patents a System That Teaches Robotaxis Which Tiny Moving Objects to Ignore
A plastic bag blowing across an intersection is not the same as a child running into the road, but a self-driving car's sensors see both as 'unidentified moving object.' Zoox has filed a patent for a system that sorts these cases out automatically.
How Zoox decides what counts as 'not in the way'
Imagine a robotaxi creeping to a halt every time a windblown coffee cup crosses its path. That's not a hypothetical problem: self-driving cars collect sensor data that turns the world into millions of unnamed blobs, and the car has to decide, in real time, which ones are threats.
Zoox's patent describes a system that automatically stamps certain sensor blobs with a label that essentially means 'this thing is small, moving, and not going to block us.' The car then drives through or around it without slowing to a crawl. The system checks the blob's size, how it's moving, whether something is blocking part of it from view, and how solid-looking it is before assigning that label.
The key safety filter is location: the system only applies this 'safe to proceed' label to objects sitting inside areas the car is already allowed to drive through. If the blob is somewhere it shouldn't be, or if it's too big or moving the wrong way, it gets flagged for normal caution instead.
How the lidar segment classifier earns the 'non-impeding' label
The patent centers on a process that automatically assigns labels to unlabeled lidar segments (lidar is the spinning laser sensor that maps a car's surroundings in 3D; a 'segment' is a cluster of laser-return points that the system has grouped together as a possible object).
For each unlabeled segment, the system runs through a checklist of criteria:
- Size: Is the object small enough to fit the profile of a low-risk item (a cone, a cardboard box, a small animal)?
- Motion: Is it moving, and in what direction and at what speed?
- Occlusion: Is part of the object hidden from the sensor, suggesting it might be larger than it appears?
- Solidity: How dense and well-defined is the laser return? A solid object reflects differently than a loose pile of debris.
If the segment passes those tests and sits inside a drivable region (an area the car's map says it can travel through), the system assigns a 'non-impeding object label.' That label tells the vehicle controller it can continue moving without treating the object as a meaningful obstacle.
The whole pipeline is designed to run automatically on raw sensor data, removing the need for a human annotator to tag every ambiguous object the car encounters during operation.
What this means for robotaxi behavior in busy city streets
Self-driving cars are notoriously cautious, and that caution has a cost: robotaxis that brake or swerve for every piece of litter or blowing leaf frustrate passengers and create unpredictable behavior for other drivers. A system that can confidently dismiss genuinely harmless objects in real time is worth real engineering effort.
For Zoox, which is building a purpose-built robotaxi for Amazon, smoother handling of ambiguous small objects is directly tied to ride quality and operational efficiency. This patent also points to a broader challenge in the field: the gap between what a sensor sees and what a human would instantly recognize as 'just a bag' is still large, and closing it through automated labeling rather than human review is the only path to scaling a fleet.
This is genuinely useful work, not a flashy patent. The 'false obstacle' problem costs robotaxi operators money and passenger trust every day, and automating the classification of harmless small objects is a real step toward fixing it. The approach is methodical rather than novel, but methodical is exactly what you want in a safety-critical system.
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.