Sony Files Patent to Track Your Body Between Camera Frames
A regular camera only takes snapshots, so quick moves can fall between them. Sony's patent adds a second sensor that never blinks, to fill the gaps.
What Sony's two-camera body tracking does
You're swinging a virtual sword in your living room, and the character on screen lags a beat behind your arm. Part of the reason is simple: a camera captures a fixed number of pictures per second, and anything that happens between two pictures is invisible to it.
Sony's filing pairs that ordinary camera with a second kind of sensor that reports changes in light the instant they happen, pixel by pixel. The regular camera finds your joints (head, shoulders, elbows, knees and so on) with good accuracy. The second sensor then follows where those joints move until the next picture arrives.
The two feeds are matched by timestamp so the system knows which sensor moments line up with which photo.
detect an attention point in a detection target on a basis of an output signal of a first sensor and one or more event signals, wherein the one or more event signals are generated in temporal correlation with the output signal of the first sensor …
Translation: The system finds points of interest by combining standard sensor data with time correlated events.
How the camera and motion sensor trade off
The patent describes a system with two sensors. The first is a normal image sensor (an RGB camera) that scans every pixel at a fixed rate. The second is an event-based sensor, a chip where each pixel only sends a signal when the light it sees changes noticeably. Because still scenes produce no signals, it can react very quickly and uses little power.
The software finds an attention point, which in the main example is one of 17 joints on a person's body, using the camera image and a trained model (software taught from labeled pictures). It draws a small box, an interest region, around each joint. Between camera frames, it reads the event signals inside that box and calculates optical flow (the direction and speed a spot is moving) to update where the joint is.
A few more details are in the claims:
- Camera frames and event signals carry matching timestamps, and the system can use event signals from just before a frame.
- It counts events per region and only updates from them once the count passes a threshold.
- Each joint can be handled on its own schedule.
The filing also says the first sensor could be a depth sensor, GPS, or a motion sensor instead of a camera.
… an event-based sensor for asynchronously generating event signals when detecting a change in the intensity of light incident on each pixel …
Translation: A special camera sensor reports lighting changes for individual pixels instantly as they happen.
What this means for games and avatars
If you play games or use apps that follow your body, the main benefit would be motion that feels less laggy. The filing names rendering a person's movement onto a computer-graphics character, copying a user's motion with a robot, and game systems that act like controllers. It also says the idea could apply to vehicles or machines, not only people.
The application is early: the examiner's first review rejected the claims as filed, which is routine and usually followed by amendments. It does not mean a product is coming. Still, the filing shows Sony Interactive Entertainment working on cutting delay in motion tracking by combining two kinds of sensors.
Sony's 38th filing we've tracked in our AR glasses race watchlist since July follows a stretching VR play area and game characters in your living room.
On the ship-path question, the software side looks close. Matching timestamps, drawing a box around each joint and counting events are ordinary programming steps. Nothing in the filing requires a new kind of algorithm.
The hardware is the hurdle. Your phone, laptop or console camera is a regular camera, and the filing needs a second, specialized sensor that is aligned with the first through a setup procedure using a shared calibration pattern. A product would have to ship with both sensors built in.
The shortest route is a dedicated camera unit with both sensors inside, which the filing itself mentions. That points to a niche accessory for motion capture or VR rather than a feature that appears in everyone's living room. The idea is sensible and well explained, but it is a long way from a box on a shelf.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0332905 A1 · click any drawing to enlarge
Show the other 4 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in