IBM Patents a Time-Sensitive Memory That Stops AI Models From Answering the Same Question Twice
Every time you ask an AI chatbot a question someone else already asked, the server runs the whole calculation again from scratch. IBM thinks it has a better way.
How IBM's time-aware AI answer cache works
Every time a company's internal AI assistant answers a question like "What are today's sales figures?" it fires up a full AI model to generate the response, even if the same question came in five minutes ago. IBM's patent describes a system that catches those repeat questions and serves up the earlier answer instead of running the whole model again.
The trick is that the system knows some answers expire faster than others. A question like "What's the weather right now?" shouldn't be cached for a week, but "What were last year's earnings?" could be reused much longer. The system reads the question, spots any time-sensitive language ("today," "current," "right now"), and sets an expiration timer accordingly.
When a new question comes in, the system checks whether it means the same thing as a recently answered one, even if the wording is slightly different. If it does, and the cached answer hasn't expired yet, your question gets answered instantly, without touching the AI model at all.
… determining, via natural language processing performed via a computer, that the first query includes a time-relative expression; storing a cache entry comprising the response to the first query and a time-to-live; …
Translation: The system checks if a question depends on time and saves the answer with an expiration limit.
How IBM times the cache window for each stored answer
The patent describes a caching layer that sits between users and an AI model, intercepting queries before they reach the model.
When a query arrives and the AI model processes it, the system uses natural language processing (software that reads and interprets human language) to check whether the question contains what the patent calls a time-relative expression: words or phrases that anchor the question to a specific moment in time, like "today," "currently," or "this week." Based on that analysis, it assigns the stored answer a time-to-live (essentially an expiration date, the way milk has a use-by stamp).
The cached answer then waits. When a new query arrives, the system runs semantic analysis (comparing meaning, not just word-for-word matching) to decide whether the new question is similar enough to the old one. If it crosses a similarity threshold and the cache entry hasn't expired, the system skips the AI model entirely and sends back the stored answer directly.
- Detects time-sensitive language in incoming queries
- Sets expiration windows proportional to how time-sensitive the topic is
- Matches new queries by meaning, not exact wording
- Returns cached answers for sufficiently similar, still-valid queries
… semantically analyzing the second query to determine that the second query has a similarity to the first query that exceeds a threshold value; retrieving the response from the cache entry; …
Translation: It compares a new question to past ones and reuses the saved answer if they match closely enough.
What smarter AI caching means for cloud computing costs
Running a large AI model is expensive in both electricity and computing time. For organizations fielding thousands of similar queries per day, the bill adds up fast. A caching layer that skips redundant model calls could cut a meaningful share of that cost, particularly in enterprise settings where employees often ask the same operational questions repeatedly.
The time-aware piece is what makes this different from simpler caching approaches. Most caches either expire everything on the same schedule or require manual rules. By reading the question itself to judge how time-sensitive the answer is, IBM's long investment in enterprise AI infrastructure this system can make that judgment automatically, which is the only practical way to handle the variety of questions a real AI assistant faces.
That makes this IBM's 29th filing we've tracked under AI assistant & agent since May, a run that includes one predicting flight diversions and one on self-repairing software builds.
The problem this patent addresses is real and not trivial. Running AI inference, the process of generating an answer from a large language model, is one of the most computationally expensive things a server does. Organizations deploying AI at scale are already paying significant infrastructure bills, and a large fraction of those queries in any enterprise deployment are functionally identical.
The solution is sensible but narrow. Semantic caching for AI queries is an active area of engineering; what IBM adds here is the time-relative detection layer, the idea that the system should read temporal cues in the question to set appropriate expiration windows. That's a practical refinement, not a conceptual breakthrough.
Whether this moves the needle depends almost entirely on how good the similarity threshold and the time-to-live logic are in practice. Set the threshold too loose and you serve stale or wrong answers; too strict and the cache rarely fires. The patent describes the architecture but leaves those tuning decisions as implementation details, which is where the real difficulty lives.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
5 drawing sheets from US 2026/0268164 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →