Adobe Patents a Two-Step AI That Pulls Visuals Into Written Answers
Most AI answers give you words. Adobe is patenting a system that goes back to the source document, finds the chart or image that belongs with those words, and weaves it into the reply.
How Adobe's answer tool finds and adds the right visuals
A researcher asks an AI a question about a dense product manual. The AI writes an answer, but the answer is just text, while the manual is full of diagrams, tables, and screenshots that would make everything clearer. You end up having to go find those visuals yourself.
Adobe's patent describes a system designed to fix that. After an AI model drafts a text answer, a second process compares pieces of that answer to chunks of the original documents it drew from. When it finds a match, it looks at what images or other media sit near that matched section, then checks whether those visuals are actually relevant to what was asked. If they are, they get folded into a new, richer answer that the first AI writes using both the text and the visuals as input.
The result is a response that can include the right diagram or table, not just a paragraph that vaguely references one. For anyone working with technical documents, reports, or design files, that difference matters.
… causing the second machine learning model to determine a first multimodal data associated with the section to include in a second response based on a similarity between a set of context information associated with the multimodal data and the first text snippet …
Translation: The system finds images or charts that match the text in its initial answer to provide better visual context.
Inside Adobe's attribution-then-retrieval pipeline
The patent describes a pipeline with two distinct AI models working in sequence.
Step one: draft and trace. A first model generates a plain-text answer to the user's question. A second model then performs source attribution (tracing which sentences in the answer came from which sections of the source documents). It does this by matching short text snippets from the answer against snippets extracted from a document collection that contains multimodal data, meaning text alongside images, charts, tables, and other media.
Step two: retrieve and rank visuals. Once the matching section of the document is identified, the system looks at the multimodal content sitting in that section. It then measures contextual similarity (how closely the surrounding context of an image or chart relates to the specific snippet being answered) to decide which visuals are worth including. Only relevant ones make the cut.
Step three: re-prompt and generate. The system builds a new prompt that packages the original draft answer together with the selected visual and its context, then hands that combined prompt back to the first model. The first model uses all of it to produce a final, multimodal response.
- Two separate ML models: one for generation, one for attribution and retrieval
- Text-snippet matching as the bridge between answer and source
- Similarity scoring to filter which visuals actually belong
- A re-prompting loop that produces the final enriched answer
… a machine learning model generates a text answer to a query, the text answer snippets are compared to text snippets from a source document to determine attribution. Based on the attribution, multimodal data is obtained from the document and used to generate a prompt …
Translation: The AI writes a text answer, checks its sources, and then pulls relevant images from those documents to create a final reply.
What this means for AI answers built on rich documents
Adobe's core customers, creative teams, marketers, legal departments, and enterprise knowledge workers, deal with documents packed with visuals every day. A plain-text AI answer pulled from a PDF report or a design brief often strips out exactly the context those visuals carry. This patent points toward a version of Adobe's AI tools where the answer you get looks more like a page from the original document than a bare paragraph.
The architecture here is pure software: no new hardware required, and the building blocks (retrieval-augmented generation, snippet matching, re-prompting) are already standard in the AI industry. That makes the path from patent to product relatively short, though the quality of the similarity-scoring step will determine whether the system actually picks the right visuals or just nearby ones. Adobe's filing sits in a growing category of new Big Tech patents aimed at making AI document tools respond with the full richness of the source material, not just its text layer.
The pipeline Adobe describes here is close to shippable. Every component already exists in commercial AI systems: retrieval-augmented generation, text-chunk matching, and multi-turn prompting are all deployed technology. The new piece is the two-pass design that uses attribution as the retrieval signal for visuals rather than treating images as a separate search problem. If the similarity scoring is tuned well, this could slot directly into Adobe Acrobat AI or Experience Manager without major infrastructure changes.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0236684 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →