Google Patents Tracking That Lets You Read Your Phone Inside a VR Headset
Put on a VR headset and your phone vanishes, along with every text that lands on it. Google's patent describes a way to draw your real phone inside the virtual scene, right where your hand expects to find it.
What Google's phone-in-VR tracking actually does
You're deep in a VR game when your phone buzzes on the desk. With the goggles on, you can't see it, and pulling them off breaks the spell.
Google's patent describes a way to bring the phone into the virtual world instead. The headset works out where your real phone is and which way it's tilted, then draws a copy of it in the same spot in the virtual scene, with your messages on its screen. You reach out, and the virtual phone is where your hand expects it.
To pull this off, the headset blends three clues: camera-based distance estimates, short-range radio signals, and motion sensors. A trained AI model turns them into a position. Older approaches needed a QR code on the phone.
inputting, into a machine learning engine at a refresh rate of a virtual environment, ultra-wideband (UWB) data, and inertial measurement unit (IMU) data, the UWB data including samples of a UWB signal acquired at a first rate, the IMU data representing samples of a IMU signal acquired at a second rate, the first rate being different from the second rate; …
Translation: The system feeds phone sensor data running at different speeds into a machine learning model to track it.
How the headset blends radio, motion and depth
The headset reads three kinds of signals to work out where your phone is and how it's turned. Depth images (pictures that show how far away things are) can come from a regular camera frame or a laser-style sensor. Ultra-wideband (UWB, a short-range radio technology that gives distance and angle) and an inertial measurement unit (IMU, the motion sensors that tell a device how it's moving and tilting) supply the rest.
The signals arrive at different speeds. In the example, depth frames come in 30 times per second, UWB readings 10 times, and motion readings 100 times. Each time the virtual world redraws (for instance, 60 times a second), the newest readings go into a machine learning engine (software trained on examples to make predictions), no matter when each reading arrived. The engine outputs three numbers for position and four numbers, called a quaternion, for orientation.
The claims cover more than one combination:
- UWB plus motion data alone
- Depth images plus UWB plus motion data
During training, a QR code shown on the phone's screen serves as the answer key for the true position and orientation. Once the engine has a pose, the headset draws a virtual phone there and can show the phone's streaming content, such as a text or email, on it.
Techniques of rendering an image of a mobile device in a virtual environment include a combination of depth images, ultra-wideband (UWB) signals, and inertial measurement unit (IMU) signals to track the position and orientation of a mobile device.
Translation: Cameras and wireless sensors work together to figure out exactly where your phone is located in virtual reality.
Why this matters for reading texts in VR
The practical payoff is a headset that lets you use your phone without leaving the virtual world. The filing describes reading texts and email on the virtual phone, and using the phone as a game paddle you swing around. It also describes tracking a fingertip within about 2 cm of the phone, so you could draw on its screen.
Accuracy is the hard part, since a floating copy that sits a few inches off the real phone is useless for reading or tapping. Google says combining the three kinds of tracking improves calibration and even lets the system calibrate itself. This is a patent application, so it shows what Google has put on paper, not what ships in a headset.
Google's 45th filing we've tracked since May in the AR glasses race adds to earlier work like cutting the math behind 3D sound and showing two images at once.
Being cut off from your phone is a small but real annoyance. Anyone who has pulled off a headset to check a text knows how it breaks the moment. A notification you can't see is one you either ignore or answer by quitting your game.
The approach is sized sensibly for that problem. Three sensors cover for each other's weak spots, and the phone needs no sticker on the back. Accuracy decides whether this works, because reading a message on a floating screen only works if the copy sits exactly where the real phone is.
The filing gives no accuracy figures, so it's hard to say how well this holds up outside a lab. The problem is modest, and the answer is a fair match for it.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0310282 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in