Sony Patents a Method to Build 3D Head Meshes from a Single 2D Photo

By Patentlyze Team · Updated May 22, 2026

Sony Interactive Entertainment has patented a way to generate a full 3D head mesh from a single 2D photo — without requiring any manual labeling of facial features first. That 'annotation-free' part is the quietly clever bit.

How Sony turns a flat photo into a 3D face model

Imagine you want to put your actual face into a video game. Today, that usually means either scanning your head with special hardware or spending hours tweaking sliders in a character creator. Sony is working on a shortcut.

The idea is to take a single flat photo of your face and automatically generate a fully-formed 3D head model from it. The system doesn't require you to manually tag where your eyes, nose, or mouth sit — it figures that out itself by borrowing structural knowledge from existing 3D face models.

The result is a hybrid: the geometry (the 3D shape and structure) comes from a reference head model, while the appearance details — what makes your face look like you — are pulled from your photo. You get a personalised 3D face without needing a 3D scanner.

How Sony blends landmark maps with 2D image features

The patent describes a three-step pipeline for generating a three-dimensional head mesh from a 2D image.

First, the system accesses a pre-established landmark correspondence map — essentially a lookup table that links specific facial anchor points (like the corner of an eye or the tip of the nose) on a source 3D face mesh to equivalent points on a target 3D head mesh. Think of it as a translator between two different 3D face templates.

Second, it takes in a plain 2D face photo showing those same facial features. No special lighting, no depth sensor, no per-image annotation required.

Third, it fuses the two: the structural geometry from the target 3D head mesh gets combined with the appearance characteristics extracted from the 2D photo. The output is a new 3D head mesh that carries:

• The head structure and topology from the 3D reference model (so it's game-engine-ready)
• The individual facial appearance from the photo (so it actually looks like the person)

The 'annotation-free' label in the title refers to the fact that the 2D image doesn't need to be pre-labeled — the system infers feature locations by leveraging the existing landmark correspondences from the 3D templates.

What this means for PlayStation avatars and game faces

For PlayStation and Sony's broader gaming ecosystem, this kind of pipeline is a direct enabler for automatic avatar creation — the kind where a player snaps a selfie and gets a usable in-game face in seconds rather than minutes. The 'no annotation' constraint is practically important: it means the system could run on-device or at scale without human review of each photo.

More broadly, any platform that needs personalized 3D avatars — social VR, virtual concerts, sports games with player likenesses — benefits from reducing the cost and friction of face digitization. If Sony can make this robust enough for consumer use, it removes one of the last awkward steps between a player and their digital self.

Editorial commentary on a publicly published patent application. Not legal advice.