Sony Patent Targets Avatar-Based Error Correction in Free-Viewpoint Sports Streams
Sony wants to stop those eerie, blurry distortions that appear when broadcasters let viewers pick their own camera angle. The fix, according to this patent, is an invisible 3D avatar that acts as a spell-checker for the video.
What Sony's avatar-based video correction actually does
Imagine you're watching a soccer match and you drag a slider to watch the winning goal from any angle you choose. That experience, called free-viewpoint video, is real and already used in some professional sports broadcasts. The problem is that the computer-generated view can go wrong: players' limbs can blur, smear, or warp when the software tries to fill in angles that no actual camera captured.
Sony's patent describes a system that checks its own work. While it builds the free-viewpoint video, it also builds a 3D stick-figure-style model of the player, poses it correctly, and renders it from the same virtual camera angle. It then compares the two images and uses the differences it finds to patch up the errors in the real video.
Think of it like a spell-checker that knows what a word is supposed to look like before flagging a typo. The viewer never sees the avatar; they just see a cleaner, more accurate video of the action.
… an avatar generation unit that generates an avatar model having a 3D shape of the subject corresponding to the posture, renders the avatar model based on the virtual viewpoint, and generates an avatar; an image comparison unit that extracts a difference between the virtual viewpoint video and the avatar …
Translation: The system creates a 3D digital twin of the athlete and compares it to the video to find visual errors.
How the avatar comparison and correction loop works
The system pulls footage from many physical cameras placed around a stadium or arena. A virtual viewpoint video generation unit stitches that multi-camera data together to produce a video from any angle a viewer or director picks, even one where no actual camera exists.
At the same time, a posture estimation unit reads the same raw camera data and figures out how the subject's body is positioned in 3D space. An avatar generation unit takes that pose and builds a 3D body-shaped model, then renders it from the exact same virtual camera angle as the main video. This avatar is essentially a simplified but correctly shaped stand-in for the person.
An image comparison unit then runs a diff (a pixel-level comparison, like tracking which cells changed in a spreadsheet) between the free-viewpoint video and the avatar image. Wherever the real video has drifted from the geometrically correct avatar shape, a correction unit applies fixes.
The result is a feedback loop:
- Capture from many cameras
- Build a 3D pose model
- Render avatar from the chosen angle
- Compare avatar to synthesized video
- Patch the video where they disagree
The image comparison unit extracts a difference between the virtual viewpoint video and the avatar. The correction unit corrects the virtual viewpoint video on the basis of the difference.
Translation: The software identifies discrepancies between the real video and the digital model to automatically fix the footage.
What this means for broadcast-quality free-viewpoint video
Free-viewpoint video is one of the few broadcast technologies that can genuinely change how sports fans experience a game. Leagues have invested heavily in multi-camera rigs for exactly this reason. But the visual artifacts that come with it, ghosted limbs, smeared faces, misshapen bodies, are a real barrier to wider rollout. A system that automatically catches and corrects those errors, without a human operator reviewing every frame, could make the technology practical for live broadcasts at scale rather than just highlight-reel packages.
Sony is well placed here. The company makes professional broadcast cameras and has deep roots in sports production infrastructure, so this patent fits neatly into existing product lines. For viewers, the payoff is an angle-any-moment replay that actually looks right. Those tracking how the imaging and broadcast industry is evolving can find plain-English patent summaries covering free-viewpoint video, spatial computing, and adjacent areas where Sony and its rivals are staking out ground.
This is the 54th Sony filing in our camera coverage we've tracked since May, building on ideas like one on lens degradation and one on fixing data errors.
The problem this patent attacks is real and underappreciated. Free-viewpoint video has been demoed at major sporting events for years, but it has never become a standard broadcast feature, largely because the output quality is inconsistent. Artifacts that look fine in a controlled demo become embarrassing in a live broadcast where millions of people are watching frame by frame.
Using a pose-matched avatar as a built-in reference image is a practical approach to a problem that is otherwise hard to define programmatically. The system gives the correction algorithm a concrete target: the video should look like this shape, from this angle. That is a more tractable instruction than asking software to detect visual errors in the abstract.
The patent is narrow in scope, covering one specific correction mechanism rather than a full free-viewpoint pipeline. Whether it is enough to move the needle on broadcast adoption depends on how large a share of the artifact problem this particular check addresses. But the underlying problem it targets is costly and real, which makes the filing worth noting even if the solution is incremental.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0253314 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →