Apple Patent Addresses Mismatched Lighting Issues in Combined 3D Avatar Representations
When part of your virtual self is captured in a bright room and the rest in a dim one, the result looks like a bad Photoshop cutout. Apple is patenting a way to fix that automatically.
Why your virtual avatar looks weirdly lit on video calls
Imagine you're on a video call and your camera captures your face in a sunny room, but software fills in the rest of your body using a model captured earlier in a dimly lit studio. The two halves would look like they belong to different people, because the light hitting each one came from totally different places.
Apple's patent tackles this by automatically adjusting the lighting on whichever part of you was captured under different conditions, so the final combined image looks like it was all lit by the same source. Think of it like a photo editor color-matching two layers, except it's happening in three dimensions and in something close to real time.
This matters most for mixed or augmented reality experiences, where your digital representation needs to blend naturally with the virtual environment around you. If the lighting doesn't match, the illusion breaks down fast.
How Apple strips and rebuilds lighting across two body captures
The patent describes a two-step lighting correction process applied when assembling a combined 3D user representation from multiple sources that were captured under different lighting conditions.
First, the system takes a second body capture and runs it through a delighting step. Delighting means stripping out the lighting information baked into an image, leaving behind what the surface of a person actually looks like without any particular light shining on it. Think of it as peeling off the shadows and highlights to get to the underlying texture.
Then the system takes that lighting-stripped version and applies relighting based on the conditions present in the first capture. This gives both halves of the combined representation the same virtual light source, so they look consistent when merged.
Key components involved include:
- A first user representation captured in one physical environment with known lighting
- An initial user representation captured in a second environment, which goes through the delight-then-relight pipeline
- A lighting representation of the second environment, used to guide the delighting step accurately
- A final combined user representation assembled from both corrected sources
What this means for FaceTime, Vision Pro, and AR presence
For Apple Vision Pro and any future spatial computing product, avatar fidelity is a central design goal. Apple has already shipped Persona, its photorealistic avatar system for FaceTime in visionOS. The persistent weak point of any composited avatar is lighting consistency: a face captured live in one environment stitched to a body model from another always carries visible artifacts that break the sense of presence.
This patent suggests Apple is building the infrastructure to handle mixed-capture avatars at a system level, where different body regions can come from different sources and different moments without looking mismatched. If this ships in a future visionOS or FaceTime update, it would raise the baseline quality of every spatial video call.
This is unglamorous but genuinely important plumbing. Lighting consistency is one of the main reasons photorealistic avatars still look uncanny in practice, and Apple is clearly investing in solving it at a foundational level rather than papering over it. Anyone tracking Apple's spatial computing roadmap should note this one.
The drawings
14 drawing sheets from US 2026/0212594 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.