Sony Patents a Shortcut That Lets a Moving Camera Map a Room Faster
Teaching a camera to map a room is easy to describe and heavy to compute. Sony's latest filing tries to cut the workload by checking only a few well-chosen points on objects instead of everything in view.
What Sony's camera-mapping shortcut actually does
You're walking through your living room with a camera in hand, and a computer is trying to work out two things at once: where the camera is, and what shape the room is. The best-known way to do that compares the brightness of tiny spots across many video frames. Do it for every pixel and the math piles up so fast that keeping pace with live video gets hard.
Sony's filing takes a shortcut. The software first spots objects in the scene, like a chair or a box, then picks a small number of points on them, such as along their edges. It follows those same points from frame to frame and checks whether they look equally bright from each angle. If they don't, the estimates of the camera's position and the room's shape get nudged until they agree.
Fewer points to check means less work for the processor.
… optimizing the estimated map information by projecting a common selection point in the captured image in the three-dimensional space to multiple pieces of the captured image data to obtain a set of projection points as related points, before comparing, in terms of luminance, the projection points with each other as the related points …
Translation: The system refines the 3D map by projecting points across different camera frames and comparing their brightness.
How Sony picks which points to compare
The system takes video from a moving camera (it can be a stereo pair of two cameras) and runs two jobs. The first, the estimation process, works out the camera's position and rotation for each frame, which the filing calls its "posture," and builds a rough 3D map with a depth guess for each pixel (how far away that spot is). This is SLAM, short for Simultaneous Localization and Mapping, a method for tracking yourself and mapping your surroundings at the same time. The "direct" version of SLAM compares raw pixel brightness.
The second job, the optimization process, cleans up that map, and it is where the new idea sits:
- Recognize objects in the 3D space and estimate each one's 3D shape.
- Pick a selection point on a recognized object, for example along an edge, or in an area with lots of brightness changes (texture).
- Project that point into each key frame (a representative frame saved whenever the camera has moved enough) where it shows up.
- Compare the brightness at those projected spots, then adjust the camera positions, object shapes and depth values until the mismatch shrinks. This fitting step is called bundle adjustment.
The filing also describes an optional second pass that adds a distance-error check, and a way to space the chosen points evenly. The cleanup runs separately from the live tracking, so the two don't have to wait on each other.
… recognizes an object in the three-dimensional space captured in each of the multiple pieces of image data so as to select the selection point by a predetermined method from over the recognized object captured in common in the multiple pieces of image data …
Translation: It identifies objects visible in multiple camera views to pick the exact spots used for tracking and mapping.
Why cheaper room mapping matters for Sony
If you've ever seen an AR app lose track of a table, you've met this problem. Mapping methods that read every pixel give detailed maps, but they eat processing power, which makes running them live on everyday hardware hard. Sony's filing aims to cut that load by checking a limited set of points, and it lists a home game machine among the devices that could run it.
The examiner's first review rejected the claims as filed, a routine step that is usually followed by amendments. The idea stays on the table either way. Anything that needs to know where it is and what's around it, from game systems to headsets, benefits when room mapping takes less computing.
Sony's 685th filing we've tracked since May on visual tools joins earlier work on collapsing gameplay footage to one frame and matching 3D characters from flat images.
The hardware list in this filing is short: a camera and a processor. Nothing new has to be built, so the shortest route to a product is a software update to mapping code that a company already runs.
What has to exist first is the stack underneath. The method needs a working mapper, an object recognizer that can spot things in the room, and a way to estimate each object's 3D shape, and the filing points to published research for those last two. A device that already has them could add this step with little fuss.
The catch is proof. The filing says the processing load goes down but gives no timing numbers, so nobody can tell yet how much faster it gets. It reads as a sensible, modest engineering fix, and its value depends entirely on those missing numbers.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
3 drawing sheets from US 2026/0350828 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in