Sony's Patent Lets a Remote Human Take Over When Self-Driving Sensors Fail
When one of a self-driving car's sensors goes dark, the vehicle is suddenly navigating blind in one direction. Sony's new patent describes a system that immediately recruits a remote human driver and feeds them a live picture from every sensor that's still working.
How Sony's sensor-failure handoff actually works
Every time a self-driving car rolls down the street, dozens of cameras, radar units, and other sensors work together to build a picture of the road. Any one of them can fail at any moment.
Sony's patent describes what happens next. The moment the car's software detects a broken sensor, it automatically starts beaming data from the still-working sensors to an off-site human operator. That person can then take over the wheel remotely, using a real-time view of the car's surroundings to steer it somewhere safe.
The key detail is that the car doesn't wait for a human to notice the problem. The handoff to remote control is automatic, and the data feed to the operator starts up on its own. You never have to be in the car for any of this to happen.
… execute automated driving that controls travel of the vehicle on a basis of the sensing data of the plurality of sensors and remote driving that controls travel of the vehicle on a basis of operation information related to the remote driving of the vehicle transmitted from an external device …
Translation: The car switches between computer driving and human remote control using external data signals.
How the car picks which data to send to a remote driver
The patent describes a vehicle control device built around three cooperating components.
First, a bank of sensors (cameras, lidar, radar) continuously scan the area around the car and produce sensing data. Second, a travel control unit can run in two modes:
- Automated driving, the car drives itself using all available sensor data.
- Remote driving, a human operator at an external device controls the car by sending steering and throttle commands over a network.
The third component is the communication control unit, which is the patent's central idea. During automated driving, it monitors sensor health. The moment any sensor is judged to have failed, it starts transmitting one of two things to the remote operator: either the raw data from the sensors that are still functioning, or a reconstructed picture of the vehicle's surroundings assembled from that surviving sensor data. Both options give the remote driver a usable view despite the gap.
The patent does not pin down exactly how sensor failure is detected, but the claim covers the entire downstream response: stopping automated control, opening the data link, and handing steering authority to a human.
… starts transmission of the sensing data (normal sensor data) of the sensors other than the sensor determined to have failed to the external device, or starts transmission of sensing data of the area around the vehicle generated on the basis of the sensing data (normal sensor data) to the external device …
Translation: If a sensor breaks, the car sends working data to an outside operator to help steer.
What this means for automated-driving safety systems
Self-driving safety has long focused on what happens when software makes a mistake. Hardware failure is a separate, messier problem: a sensor can stop working without warning, and the car still has to reach a safe stop. Sony's approach treats remote human takeover as the fallback layer for that hardware-failure case, which is a practical framing because remote-operation infrastructure is already being built for robotaxis in several cities.
The patent comes from Sony Semiconductor Solutions, the division that makes image sensors found in a large share of the world's cameras. That gives the filing a natural fit with vehicle camera systems. Among the new Big Tech patents covering automated-driving sensor stacks, Sony's entry into the failure-recovery space reflects how much the camera-hardware side of the industry is pushing into vehicle intelligence.
That makes this Sony's 19th filing we've tracked since July in the self-driving sensing race, a series that includes a dual-range camera and an adaptive electromagnetic scanner.
Getting this feature into a real car requires more than clever software. A remote driver needs a fast, reliable wireless connection, a staffed control center somewhere, and legal permission to operate in every city where a car might break down. This document describes none of that groundwork.
The filing itself covers a fairly straightforward idea: if a sensor stops working, start sending footage from the remaining working sensors to a human operator who can then steer the car to safety. That logic is not the hard part. Building and funding the network of people and infrastructure behind it is.
Sony's clearest near-term gain here is on paper. If its camera hardware ends up inside fleet vehicles, owning a patent on what happens when that hardware fails is a sensible defensive position, even if the full system is years away from a passenger's seat.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0249864 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →