Get the newsletter
Sony · AI / ML · Filed Mar 31, 2025 · USPTO published Oct 1, 2026 · Oct 1 release

Sony Patents a Single AI Vision System That Learns Multiple Visual Tasks at Once

Training a separate AI for every visual task, detecting objects, labeling scenes, measuring depth, is expensive and slow. Sony's new patent describes a way to build one shared visual brain that handles all of them together.

A single AI vision system uses a pretrained Vision Transformer and an adapter to perform various visual tasks like detection, segmentation, classification, and pose estimation. Drawing from patent filing US 2026/0301395 A1.
A single AI vision system uses a pretrained Vision Transformer and an adapter to perform various visual tasks like detection, segmentation, classification, and pose estimation.All 3 drawings ↓
Publication number
US 2026/0301395 A1(US20260301395A1)
Applicant
Sony Group Corporation
Filed · Published
Mar 31, 2025 · Oct 1, 2026
Inventors
Lingjuan LYU, Weiming ZHUANG, Zhizhong LI, Chen CHEN, Sina SAJADMANESH, Jingtao LI, Jiabo HUANG, Vivek SHARMA, Peter H. STONE, Michael S. SPRANGER
Status when we published
Waiting for an examiner
Document
17 claims
US classification
382/157
Examiner
ZAK, JACQUELINE ROSE (Art Unit 2666)

What Sony's multi-task vision AI actually does

You're building a camera system that needs to recognize faces, read street signs, and separate objects from backgrounds, all at the same time. Right now, most AI setups require a separate model trained for each job, which costs time, money, and a lot of computing power.

Sony's patent describes a different approach: take a large, already-trained AI that's good at "seeing" (called a Vision Transformer), freeze it so it doesn't change, and attach a small trainable add-on called an adapter. That adapter learns to translate what the big AI sees into outputs for many different tasks simultaneously.

The result is one compact system that can be trained to do multiple vision jobs at once, rather than building and maintaining a fleet of separate AIs. It's designed to be more efficient to train and potentially easier to deploy in real products like cameras, robots, or autonomous vehicles.

From the filing · CLAIM 1
a frozen pre-trained vision transformer ‘ViT’; and a trainable adapter configured to interact with tokens of the ViT for successive interaction blocks of the ViT, by use of an injector and extractor; …

Translation: It combines a locked base vision model with a flexible add-on component that modifies the data flow.

How the adapter talks to the frozen vision model

The core of the system is a Vision Transformer (ViT), a large neural network pre-trained on enormous amounts of image data. Think of the ViT as a highly experienced visual analyst who already knows how to process images deeply. In Sony's design, this ViT is "frozen," meaning its learned knowledge is locked in place and not modified further.

Attached to the ViT is a trainable adapter, a lightweight module with two working parts: an injector and an extractor. The injector feeds task-relevant information into the ViT's processing pipeline at each layer (called a "block"), while the extractor pulls out useful signals at each stage. Together they let the frozen ViT's representations be steered toward whatever tasks are needed, without altering the ViT itself.

The adapter's output feeds into a set of decoders, each specialized for a different vision task, such as semantic segmentation (labeling every pixel in an image by what it belongs to), object detection, or depth estimation. All of these decoders are trained together in a process called multi-task learning, where the adapter and decoders learn simultaneously by sharing information across tasks.

  • Frozen ViT backbone: preserves expensive pre-training, no retraining cost
  • Injector/extractor adapter: steers the ViT's attention toward multiple tasks
  • Multiple decoders: each handles one visual task, trained together in parallel
From the filing · THE ABSTRACT
… the trainable adapter is trained in conjunction with training a plurality of different decoders comprising a first set of decoders, respectively coupled to the backbone for respective tasks, to produce the completed vision foundation model based on multi-task learning.

Translation: The system learns many different visual jobs at once by pairing the core model with multiple specialized task decoders.

What this means for cameras and computer vision devices

Training large AI vision models from scratch is expensive enough that only a handful of companies can do it regularly. Sony's approach cuts that cost by reusing a frozen pre-trained model and only training the small adapter and decoders on top. That makes capable multi-task vision AI more accessible and faster to update when new tasks are needed.

For products like smart cameras, robots, and autonomous vehicles, the ability to run one shared visual model across many tasks, rather than several separate models, reduces hardware demands and simplifies deployment. a growing pile of Sony AI vision filings suggests this is part of a broader push to bring foundation-model thinking into Sony's imaging and sensor hardware lines.

Sony's 11th filing in the AI vision patents we've tracked since May follows work like one on ghost image removal and one on free-angle scene freezing.

Source: US 2026/0301395 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 3 sheets

3 drawing sheets from US 2026/0301395 A1 · click any drawing to enlarge

AI vision system architecture, drawing sheet 1 from US 2026/0301395 A1
Sheet 1 AI vision system architecture
Detailed AI vision system components, drawing sheet 2 from US 2026/0301395 A1
Sheet 2 Detailed AI vision system components
Steps for training the AI system, drawing sheet 3 from US 2026/0301395 A1
Sheet 3 Steps for training the AI system
Patent filing page

Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.