Samsung Patents an On-Device AI That Learns New Objects Without Forgetting Old Ones
Teaching an AI to recognize new things usually makes it forget old ones. Samsung's patent describes a way to solve that, right on your phone, without sending your photos to a server.
How Samsung's image-learning AI avoids the forgetting problem
Today's image-recognition AI has a frustrating limitation: every time you train it to identify something new, it tends to overwrite what it already knew, a problem researchers call "catastrophic forgetting." Samsung wants to change that for the AI running directly on your phone.
The patent describes a system where your device's AI already knows a large set of common categories, think general objects like "chair" or "car." When you want it to learn something personal, like your specific mug or your dog's favorite toy, it doesn't start over. Instead, it learns only the small difference between your new item and the closest thing it already understands, storing that gap rather than rebuilding its entire knowledge base.
The result is an AI that can recognize your personal objects while keeping everything it already knew intact, and it does all of this locally, so your images stay on your device.
… calculating a residual vector comprising a distance and a direction between the extracted input feature embedding vector and the selected prototype feature embedding vector in a feature space which is defined by input feature embedding vectors extracted for the multiple input images …
Translation: It measures the exact gap between a new image and what the AI already knows.
How residual vectors let the model learn without retraining from scratch
The system is built around two AI components working together. The first is a backbone feature embedding model, a pre-trained neural network that converts any image into a list of numbers (called a feature vector) that represents its visual characteristics. This model already knows a broad set of categories, each represented by a prototype vector, essentially an average mathematical fingerprint for that category.
When the system needs to learn a new personal object class, it doesn't retrain the whole backbone. Instead, it computes a residual vector for each training image: the mathematical distance and direction between that image's feature vector and the prototype of the nearest known category. Think of it as noting "this new mug looks like a generic cup, but shifted in these specific ways."
Those residual vectors are used to train the second component, a generative ML model (an AI that can produce new examples, similar in spirit to how image generators work). That generative model learns the pattern of residuals, so it can later synthesize new training examples for a class without needing the original photos.
- New classes are added without modifying the backbone
- Old class knowledge is preserved because the backbone is frozen
- Synthetic data from the generative model fills in gaps when real examples are scarce
- Everything runs on-device, keeping personal images private
… enable class-incremental recognition of personal object classes via conditional feature-space generative modelling with near-zero forgetting …
Translation: This allows the AI to learn new items without wiping out its old memories.
What this means for photo apps and personal AI assistants
The "forgetting" problem is one of the most practical barriers to useful personal AI on a phone. If teaching your device to recognize your cat ruins its ability to tell a car from a truck, the feature is worthless. Samsung's steady investment in on-device AI points toward phones that build up a personalized knowledge base over time, the way a person does, rather than resetting with every update.
For you as a user, this could mean a photo app or personal assistant that genuinely gets better at recognizing your things over time, without requiring a cloud upload or a factory reset of its memory every time you add something new. That's a meaningful change in how useful on-device AI can actually be in daily life.
That makes this Samsung's 20th filing we've tracked since June in our on-device AI privacy watch, following work on fixing poor voice input and choosing power-saving modes on the device.
The problem this patent attacks is real and underappreciated. Catastrophic forgetting has been a known obstacle in machine learning for years, and it becomes especially costly when the AI is supposed to be personal, learning about your specific life rather than generic categories. A phone AI that forgets your dog every time it learns your car is genuinely useless as a personal tool.
The approach here is measured and technically credible. Keeping the backbone frozen and training only a small generative model on residuals is a recognized strategy for this problem, and doing it on-device adds a privacy argument that matters to a lot of consumers. This isn't a moonshot; it's careful engineering aimed at a specific, well-defined failure mode.
The open question is whether the generative model's synthetic examples are good enough to actually help in practice, especially for unusual personal objects that look nothing like any base category. That gap between a clean lab result and a messy real-world deployment is where patents like this either prove their worth or fade out.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
33 drawing sheets from US 2026/0268643 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in