Apple Patents a System to Keep Virtual Stand-Ins for Remote Callers Looking Consistent During Group Video Calls
When two people wearing AR headsets sit in the same room on the same video call, do the remote participants appear in the same positions for both of them? Apple has a patent describing exactly how to make sure they do.
How Apple keeps remote faces consistent for everyone in the room
Today, if you and a colleague are both wearing AR headsets in the same conference room and join a video call, there is no guarantee that the floating virtual faces of your remote teammates appear in the same spots for each of you. That inconsistency can make it awkward to agree on who just spoke or where to direct a shared gesture.
Apple's new patent tackles that problem by letting the two headsets in the room coordinate. When something changes, like a remote participant getting promoted to a more prominent position because they started talking, one headset checks whether the change is significant enough to bother the other headset about. If it is, both devices update their layouts together. If it is not, each device holds its current view.
The goal is that both you and your seatmate see the same arrangement of remote people, so a shared gesture or glance across the table actually lands on the same virtual face for everyone.
… maintaining visual consistency of a visual representation of one or more remote users in a three-dimensional environment within a multi-user communication session that includes a group of collocated users.
Translation: Keeping remote callers looking the same when multiple people share the same room.
How the sync decision gets made and sent to nearby devices
The patent describes a multi-device coordination method for mixed-reality video calls that involve at least two people physically sharing a space, each on their own headset or AR device, while also connected to remote participants.
Here is how the core logic works:
- Prioritization data (a ranked score that determines which remote participants get more prominent display positions) is tracked continuously by the local devices.
- When that score changes, such as when audio activity, speaker focus, or some other signal shifts, the device evaluates whether the change clears a threshold defined by one or more criteria.
- If the threshold is met, the device recalculates the layout, updates its own display, and sends matching display data to the co-located device so both show the same new arrangement.
- If the change is too small to clear the threshold, the device holds the existing layout and sends nothing, avoiding constant churn and visual distraction.
The patent also covers a related transition: moving a remote participant's representation from a flat virtual tile (think a floating video rectangle) into a full three-dimensional avatar within the shared environment. That shift can happen dynamically during a call as conditions change.
The filtering step, where small prioritization changes are ignored, is important. Without it, the layout would reorganize constantly and the coordination packets would flood the local network.
updating presentation, via the one or more displays, of the plurality of virtual representations to have a second spatial arrangement, different from the first spatial arrangement; …
Translation: Rearranging where people appear in the virtual room based on new rules.
What this means for shared AR meeting spaces
For AR meetings to feel like real shared spaces, everyone in the room needs to perceive the same social geography. If your colleague's headset shows the remote CEO front and center while yours tucks that person off to the side, eye contact breaks down and spatial cues stop working. Apple's coordination logic is an attempt to solve that before it becomes a common complaint.
The filtering mechanism also hints at a practical design choice for Vision Pro-style environments where local Wi-Fi or wired connections carry the sync traffic. Sending a full layout update for every tiny priority fluctuation would waste bandwidth and cause visual jitter, so the threshold acts as a gatekeeper. This filing sits alongside the broader wave of latest Big Tech patents aimed at making shared AR spaces feel socially coherent rather than just technically connected.
Apple's 33rd filing we've tracked in our AR glasses watch since May continues the color-correction thread seen in the borrowed-camera fix and the on-screen reading approach.
The most interesting design choice here is that the headsets do not constantly talk to each other. Instead, each one waits until a priority change is large enough to be worth reporting, then sends an update. That saves a flood of tiny messages, but it also means two headsets can briefly fall out of step during a fast conversation, and that lag could feel odd to the wearer.
The trade-off is probably the right call. Constant small updates would create their own mess. The problem is that Apple never says in this patent exactly how big a change has to be before it qualifies as worth sending. That missing detail matters enormously. The real user experience will depend entirely on how engineers set that threshold when they actually build the thing.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
26 drawing sheets from US 2026/0245312 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →