IBM Files Patent for Showing Which Words or Image Areas Swayed an AI
Ask an AI why it called a sentence ungrammatical and you usually get a verdict with no reasoning. IBM's new filing describes a way to make the system point at the exact word that tipped the call.
What IBM's swap-a-word explanation tool does
One common way to find out what an AI was looking at is to delete a word and see whether the answer changes. That works for judging whether a review sounds positive. IBM's filing says it can fall apart on other jobs, like checking whether a sentence is grammatical.
IBM's patent application tries something different. Instead of deleting a word, it swaps in a believable stand-in and watches how much the AI's confidence shifts. Take 'mary wrote a letter to himself last year.' The AI calls it ungrammatical. Swap mary for he and its confidence in that call collapses, so mary gets flagged as the word that mattered most.
You would then see a short report ranking the culprits: mary first, himself second. The same idea can highlight the spot on a photo that made an AI call a factory part defective.
… generating an explanation of which of the plurality of input parameters is most relevant to the prediction based on contextually replacing respective ones of the plurality of input parameters …
Translation: The system figures out what drove the AI decision by swapping out different parts of the input data to see what changes.
How IBM swaps words to test an AI's verdict
The filing describes two models working together. The first is the AI you actually care about, such as one that judges whether a sentence is grammatical or whether a photographed part has a defect. It produces a prediction and a confidence score (a number from 0 to 1 showing how sure it is).
The second is a replacement model, which proposes stand-in values for one piece of the input at a time. For text, the filing suggests a masked language model (an AI trained to guess a missing word from the words around it). For images, it names masked object and region models. The explanation then comes together in these steps:
- Hide one word or image area and let the replacement model suggest a set number of fitting substitutes (five in the example, and a user can adjust the number).
- Rebuild the input with each substitute and run the main AI again.
- Record how far the confidence score moves from the original.
- Pick the piece whose substitute moved it the most, or rank all the pieces.
The filing says ranking can use the single biggest change, or other math such as a sum or median across a piece's substitutes. The result screen shows the task, the input, the verdict, the confidence, and the top-ranked pieces. In the factory example, a camera feeds part photos in automatically and the screen highlights the suspect area.
Why IBM's AI explanations could build trust
If an AI rejects a sentence or flags a part on an assembly line, the first question is why. An explanation that points at the exact word or image area gives a person something to check before trusting or overruling the machine. The filing's own example is a factory camera, where a highlighted patch shows a worker what looked wrong.
The filing says swapping in believable substitutes produces explanations that stay faithful to what the model actually relied on, because the altered sentence still reads sensibly. That is IBM's claim, and the filing backs it with one worked example rather than test results. The document also notes it was made with Army Research Office funding.
IBM's 75th filing we've tracked since May in the AI guardrails race builds on their self-quizzing and correction work and AI identity verification quiz, adding yet another approach to keeping AI models in check.
Claim 1 is short. It covers three steps: get a prediction from a trained model, work out which piece of the input mattered most by swapping in fitting replacements, and show the result.
Look at what the claim leaves out. It names no particular kind of AI, no confidence scores and no data type such as text or images. Those details only arrive in later claims: confidence scores in claim 5, deep learning in claim 8, words and image areas in claims 9 and 10. As written, claim 1 would reach any explanation tool that swaps in context-fitting stand-ins, whatever model sits underneath.
The whole weight rests on the phrase 'contextually replacing,' which claim 1 never defines. A claim this open-ended may well get trimmed back toward the narrower details in claims 5 through 10 before anything is granted. Even so, the grammar example in the description shows a concrete, working idea.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0350365 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in