Sony Files Patent to Train Eye-Tracking AI on Computer-Generated Faces
Sony wants to teach software where you are looking by showing it thousands of computer-generated people instead of real ones. The filing describes a virtual studio that tilts faces, moves a camera and rejects bad pictures automatically.
What Sony's avatar-based gaze training does
Every time you join a video call, your laptop camera sees a slightly different version of you: closer, farther, head tilted, eyes on a slide or on the wall. A gaze-tracking program has to cope with all of it.
Teaching it takes thousands of face pictures, each with the right answer attached, and real people are a poor source. Their answers are noisy, and the photos tend to bunch up, say mostly faces in the lower half of the frame. Sony's patent builds the pictures instead. A computer-generated person sits in a virtual room, a virtual camera moves around, and the system aims both of the person's eyes at a chosen target so it knows exactly where they are looking.
Then it repeats the process with new faces, ages and skin tones until the collection is varied enough.
controlling a state of a three-dimensional model of a person in a three-dimensional virtual space and a rendering condition for rendering the three-dimensional model …
Translation: Sony adjusts how a digital person looks and is displayed inside a virtual environment.
How Sony's avatars get tilted, moved and filtered
The system builds three things inside a 3D computer-graphics scene: an avatar (a computer-generated person), a camera object (a virtual camera that takes the pictures), and a gaze point (an invisible target the avatar looks at). Each eye is its own eyeball object, so the system can aim both eyes at the same spot.
Next it changes the scene step by step. The filing gives examples:
- Face tilt, nod and turn each swept from -25 to +25 degrees in 5-degree steps (11 steps per direction).
- The camera placed at five distances, with 20 face positions at each, for 100 relative positions.
- A grid of 81 gaze targets spaced 5 cm apart.
Every combination produces a rendered picture paired with its answer key. That answer is a single gaze direction for both eyes, which means the finished model does not have to reconcile two conflicting eye estimates.
The system also checks its own work. If an eye falls outside the picture, hides behind the face, or looks almost entirely white because the iris is barely visible, that picture is left out of the training set.
A third variation analyzes an existing set of real photos, finds its bias (for example, faces mostly about 50 cm from the camera), and generates avatars at 40, 60 and 70 cm to fill the gaps. The system also saves a script for each picture so it can be recreated later.
… generation processing of a learning data set of a line-of-sight estimation model, for example …
Translation: This system helps build training data to teach AI how people look when they stare at things.
What this means for webcam gaze tracking
If it works, the payoff is gaze tracking that holds up in messy real conditions. The filing's own example is an online meeting where a presenter can tell which part of a shared slide the audience is looking at and add detail where it is needed. It also says the same approach could train systems that read gestures, emotions or lip movements.
The saved scripts add a privacy angle: a dataset can be rebuilt on demand without keeping photos of real people. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. Nothing here names a Sony product, so treat it as a look at how the company prepares training data.
Sony's 39th filing we've tracked since July in the AR glasses race adds to a body of work that includes applications like body tracking across frames and a speed-based play area.
Training on made-up people buys clean answers and costs realism. A rendered eye is tidy, so a model taught only on avatars could stumble on the smudged, squinting, badly lit faces it meets in real life. Sony's filing seems aware of this, since it lets real photos and avatar images mix and matches the avatar pictures to the format of the real ones.
The rule that throws out bad images also trims the training set. A face turned hard one way with eyes rolled the other is a rare pose, and the model never practices it. Those rare poses could be exactly where it fails.
The trade still looks worth it. Perfect answer labels and full control over the spread of faces beat noisy photos of volunteers, and any gap shows up in testing, where it can be measured and patched.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
27 drawing sheets from US 2026/0330788 A1 · click any drawing to enlarge
Show the other 19 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in