Samsung Files Patent to Stop AI From Making Up Details About Photos
Samsung describes a way to have an AI write several captions for a photo, rank them, and learn from the best and the worst, all without new photos or paid outside help.
What Samsung's photo-caption patent teaches AI
Ever asked an AI to describe a photo and had it confidently mention a dog that isn't in the picture? That's a hallucination: the AI states something false as if it were fact, and it's one of the biggest reasons people hesitate to trust these tools.
Samsung's filing describes a way to teach an image-reading AI to catch itself. Instead of hiring people to grade answers, or paying another company's AI to do it, the system has an AI write several different descriptions of the same photo. A second tool then scores how well each description matches the picture. The best-scoring description becomes the "right" example, and the worst one becomes the "wrong" example.
Those pairs are then used to retrain the AI so it leans toward accurate answers and away from believable but false ones. The photos are ones the AI was already trained on, so nothing new has to be collected.
Each image in the training dataset comprises two captions-a positive caption (or label) and a negative caption (or label). The goal is to fine-tune the pre-trained VL ML model using this training dataset such that the model learns to produce outputs that are more like the positive captions and less like the negative captions.
Translation: Training photos use one good description and one bad one to teach the AI what to avoid.
How the AI picks its best and worst captions
The filing covers a method for building a training dataset (a collection of examples an AI studies to improve) that is used to fine-tune a vision-language model (an AI that handles both pictures and text, such as one that captions photos or answers questions about them).
For each image, the method runs these steps:
- A pretrained vision-language model writes several captions for the image.
- Each caption gets a score for how well it matches the image. The filing's example uses CLIP, a model trained to measure image-text similarity, which compares embeddings (lists of numbers that capture meaning) using cosine similarity.
- The captions are ranked, and the top and bottom ones become the positive and negative labels.
- The image and its label pair are saved to the dataset.
The filing also describes using at least two different prompts to get varied captions, and a route where a language model turns a caption into a question with one correct answer and one believable wrong answer.
Filters keep the pairs useful. Captions below a score threshold (28.0 in one example) are dropped as too obviously wrong, overly long captions are removed, and pairs of different lengths are discarded so the AI doesn't learn that short means bad. The pair with the biggest score gap is kept, and training then pushes the model toward the positive caption.
… selecting, from the ranking, a pair of text captions comprising the most accurate text caption and the least accurate text caption; assigning the selected most accurate text caption as a positive label for the image and assigning the selected least accurate text caption as a negative label for the image; …
Translation: The system picks the best and worst descriptions of a photo to use as training guides.
Why cheaper AI grading matters for photo chatbots
If you ask a chatbot about a screenshot, a webpage or your own notes, you want an answer that matches what is on the screen. The filing mentions exactly those uses: a visual chatbot that answers questions about an image or a page, and a tool that searches through notes and slides that mix pictures and text. Fewer invented details would make those tools easier to trust.
The cost angle may matter just as much. The filing faults older approaches for needing extra data, several models and a paid GPT-4 service to rank answers. Samsung's version reuses photos the model has already seen, and it reports fine-tuning on about 0.7 million samples. Those results come from Samsung's own testing, so treat the comparisons to larger models as claims for now.
That makes this Samsung's 13th filing we've tracked in our AI safety checks race since June, adding to earlier applications like one checking car alerts and one fact-checking voice answers.
Made-up details are a real problem for AI that looks at pictures. If a tool misreads a receipt, a medicine label or a webpage and invents a detail, you may act on something false, and the answer usually sounds just as sure as a correct one.
The approach fits the size of that problem. Grading AI answers has been expensive, because it takes people or a paid outside AI, and Samsung's method has the model's own captions do most of the work on photos it has already seen. Cheap grading can be repeated across many models and updates, which matters for a flaw this widespread.
Hallucinations will probably persist after this kind of training, and the reported gains come from Samsung's own tests. Still, a low-cost way to chip away at a costly flaw is a sensible match for the job.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
14 drawing sheets from US 2026/0341002 A1 · click any drawing to enlarge
Show the other 6 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in