Adobe Patents a System That Teaches AI When to Say 'I Don't Know'
AI assistants that confidently make up answers are a well-documented headache. Adobe's new patent is specifically designed to train an AI to recognize when a question is simply outside its lane.
How Adobe stops product AIs from answering off-topic questions
Right now, AI-powered customer service tools often answer product questions even when they should not, because they have no reliable way to tell whether a question is actually about the product they were trained on. Adobe wants to build a gatekeeper that solves that.
The idea is to train a separate, smaller AI model whose only job is to judge incoming questions. Before a question ever reaches the main product AI, this gatekeeper decides: is this question something the product AI was actually built to handle? If not, the question gets blocked or redirected, rather than answered with a confident but wrong response.
To train that gatekeeper well, Adobe's system generates its own practice questions automatically, at varying levels of specificity, from very detailed to quite broad. That range helps the gatekeeper learn to recognize out-of-scope questions in many forms, not just the obvious ones.
… training, by the processing device, a classifier machine-learning model to control whether processing of a product query is permitted by a product machine-learning model associated with the product …
Translation: The system trains a secondary AI to act as a gatekeeper that decides if the main AI is qualified to answer a specific question.
How the classifier learns to spot out-of-scope product queries
The patent describes a query scope control system built around a classifier model, a secondary AI whose job is not to answer questions but to judge whether a given question is answerable by the primary product AI.
The key engineering challenge is generating good training data for that classifier. The system solves this by prompting a large language model (an AI that generates text) to produce synthetic questions from a product's documentation. It then prompts the same model again to restate each question at a higher level of abstraction, meaning a broader, more vague version of the same inquiry. This produces a training dataset that covers both specific and general phrasings of in-scope and out-of-scope questions.
A sampling technique is also applied during training data generation to keep the dataset balanced, preventing the classifier from being skewed toward one type of query.
The classifier is then trained on:
- User-generated queries (real questions people have asked)
- Machine-generated queries at multiple abstraction levels
- Ground-truth labels indicating whether each query is in-scope for a given product
The result is a model that can intercept incoming questions and decide, before any product AI processes them, whether answering is appropriate.
The question answering system also supports an ability to generate machine-generated queries with increasing levels of abstraction that further supports comprehensive training data.
Translation: The system creates its own practice questions at different levels of complexity to help the AI learn more effectively.
What this means for AI assistants built on product data
AI assistants tied to specific products, whether a software help bot or a retail chatbot, frequently fail in two directions: they refuse questions they could answer, or they answer questions they have no business touching. The cost of the second failure is higher, because it produces wrong information delivered confidently. Adobe's approach puts a dedicated filter in front of the product AI, which is a more targeted fix than trying to teach the main AI to know its own limits.
For companies building AI on top of product documentation, this kind of scope control is a practical necessity, not a luxury. Adobe's focus on automatically generating training data at varying specificity levels is the part worth watching most closely, since hand-labeling thousands of in-scope and out-of-scope questions is expensive and slow. This filing sits alongside a steady stream of new Big Tech patents targeting the infrastructure problems of enterprise AI, specifically the gap between what these systems claim to handle and what they actually handle reliably.
This is the sixth Adobe filing we've tracked since July in our AI guardrails race watchlist, following ones on routing data to human reviewers and sourcing AI answers.
The problem this patent attacks is real and underappreciated. Businesses deploying AI assistants on product data face a consistent liability: the AI confidently answers questions it was never equipped to handle. That erodes user trust fast, and fixing it by retraining the main model is expensive and slow.
Adobe's approach, a dedicated classifier trained on automatically generated queries at multiple abstraction levels, is proportionate to the problem. The abstraction ladder (specific question, then progressively vaguer versions of the same question) is a practical idea for covering the full range of how real users phrase things.
The genuinely uncertain part is how well synthetic, machine-generated training questions map to the messy, idiosyncratic questions real users actually ask. That gap between lab-generated training data and real-world inputs is a known weak point in this class of systems. Adobe's inclusion of user-generated queries in the training mix suggests they are aware of it, but the patent does not resolve it.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0253129 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →