Adobe Patents an AI Chat Tool That Answers in Both Words and Pictures
Ask a chatbot about a photo of your dog and it could answer in words and also draw you a new picture, all in a single reply.
What Adobe's words-plus-pictures AI reply does
Adobe's filing starts from a problem: today's text-focused AI models struggle when you hand them a picture along with your words. Show one a photo of a dog with the message "[image] is cute" and, the filing says, it can't reliably make sense of the image. The usual workaround is to build one giant model from scratch that handles text and pictures together. That takes huge amounts of computing power, and it has to be redone whenever a new kind of media gets added.
Adobe's fix is to leave the existing language model exactly as it is and put a second, separate image model beside it. Both read the same input, and the system picks which one's output to use for each piece of the reply.
The result is an assistant that can read your photo and your message, then answer with text, a freshly generated image, or both. In the filing's example, the dog photo plus "is cute" becomes the response "dog is cute."
… generating, using a language generation model, a text token based on the input prompt; generating, using a media generation model, a synthetic media based on the input prompt, wherein the synthetic media depicts a scene corresponding to the input description; …
Translation: The tool uses one AI model to write text and another to create a picture based on your prompt.
How Adobe's two-tower setup keeps language intact
Claim 1 covers a straightforward flow. A user's prompt includes a description of some media. A language generation model (the same type of AI that powers text chatbots) produces a text token, which is a word or a piece of a word. A separate media generation model produces a synthetic image of the matching scene. The system then combines the picture with the text into one reply.
The filing's example design is a fusion model that runs two "towers" (parallel stacks of processing layers) side by side:
- A text tower built from a pretrained language model whose weights (its learned settings) are frozen, so its language skills stay as they were.
- A vision tower that starts as a copy of the language layers and is trained to handle images. Both towers read the same input at every layer.
- A step that takes each piece of the output from whichever tower matches it, text or image.
The image side is a diffusion model, which makes a picture by starting from pure noise and removing it step by step. Training combines a text-prediction error score with an image-denoising error score, and lines up the vision features with a pretrained image encoder called CLIP to speed up learning.
In the filing's tests, built on Llama-3, the two-tower design reached a 23% lower FID score (a standard image-quality measure where lower is better) than existing models, using the same number of training parameters.
A response to the input prompt generated based on the text token and the synthetic media is presented to the user.
Translation: The system then shows you the final answer, which mixes both the written words and the new image.
What Adobe's patent means for AI chat and image tools
For you, the payoff would be a single assistant that reads your photo and your words, then answers with text and a newly drawn image instead of sending you to separate tools. The filing's own example is a dog photo and the message "is cute" turning into a finished reply.
For Adobe, the practical angle is cost. The filing says building one model from scratch for every media type eats computing power and forces retraining each time a new type is added. Keeping the language model frozen and adding one tower per media type, such as audio or video, could avoid that. Whether Adobe turns this into a product is not something the patent says.
Adobe's 31st filing we've tracked since May on controllable AI images joins earlier work on room lighting from one photo and fur texture on clothing.
Claim 1 is short and wide. It asks for a prompt that describes some media, a text model that produces words, a separate image model that produces a picture of that scene, and a reply that puts the two together.
Nothing in that claim mentions the frozen language model, the paired towers or the noise-removal image method that fill the description. Those details appear only in later claims, such as the one naming a diffusion model. Read as written, the opening claim would reach any setup that writes text and draws a matching picture in one reply, however the two parts are wired together.
A claim this wide invites an obvious question: can it survive review? The filing does not answer that. If it does, the specific design Adobe describes lives in the dependent claims, and those are where the real engineering is.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Adobe patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
16 drawing sheets from US 2026/0310894 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in