IBM Files Patent to Make AI Answer Faster While Your Data Stays Encrypted
Running an AI on data that stays encrypted the whole time is painfully slow. An IBM patent application tries to fix that by letting a small, fast model draft the reply while a big model checks it.
What IBM's draft-and-check AI trick does
A junior writer drafts three sentences in a minute, and a senior editor checks all three at the same time. Every sentence that passes goes straight into the story, and the writer picks up from there.
That is the basic trick in this IBM patent application for making AI text generators faster. A small, quick AI model guesses the next few words of a reply. Several copies of a big, slower model then check those guesses all at once. Guesses that match are kept, and the small model keeps going from there.
The twist is that all of this happens on encrypted data, the kind where the computer does the math without ever seeing your actual words. That level of privacy normally makes AI painfully slow, so IBM's filing is about winning some speed back without giving up the lock.
… comparing each token produced by the small language model to a corresponding LLM token; concatenating each LLM token that resulted in a match with input tokens of a current context input; …
Translation: It checks where the fast small model agrees with the slow large model and combines those matching parts.
How IBM checks AI guesses without seeing them
The filing covers a draft-and-check loop for generating text. A small language model writes a batch of k tokens (tokens are the word-sized chunks AI models read and write, and k is how many get drafted at a time). Then several copies of a large language model each check one of those spots at the same time, so the slow model does not have to write one chunk after another.
Each drafted token is compared with the large model's token for the same spot. The result is a list of ones and zeros, such as [1, 1, 0, 0] when the first two guesses were right. The filing bumps that to [1, 1, 1, 0], because the big model's answer at the first miss is still correct. The verified tokens are added to the running text, and the small model drafts the next batch.
Two details handle the encrypted setting:
- This type of encryption only allows addition and multiplication, so picking the most likely word and comparing guesses are done with polynomial math (formulas built only from those two operations) instead of if-else decisions.
- The batch size k is retuned after every round, based on how many of the small model's guesses were right.
The first claim covers the draft-and-check loop itself. Running it in an encrypted environment appears in a narrower claim.
Text is generated by predicting, using the small language model, an additional k tokens based on the concatenated tokens.
Translation: The system finishes writing the response using the smaller, faster model with the verified text.
Why encrypted AI needs a speed boost
Say you want to ask an AI about medical notes, a legal dispute or your finances without the company running the AI reading any of it. Encryption of this kind promises exactly that. The catch is speed: the filing's own background says AI models already take a long time to respond, and doing the work on encrypted data makes the wait longer still. Cutting that wait is what would make the idea usable.
The filing says the method keeps the same accuracy as the original large model, because the big model checks every word that gets kept. It is a patent application, not a product, and the text gives no speed numbers, so how much faster this gets is still unknown.
That makes this IBM's 22nd filing we've tracked since May in our AI models working in teams watch, which already includes their work on an AI referee picking answers and AI models competing to respond.
Privacy-preserving AI has a cost problem that anyone who has waited on a slow chatbot can imagine, multiplied many times over. The filing's own background says AI text models already respond slowly and encrypted versions respond more slowly still, which leaves a hospital or bank little reason to switch the privacy feature on.
IBM's answer cuts the number of slow steps. A small model drafts several words, and the big model checks them together, so fewer rounds of heavy encrypted math are needed. That matches the problem well, because the waiting is what hurts.
The weak spot is proof. The text offers no timing results, and the extra copies of the big model add their own computing bill. Until numbers appear, this is a sensible fix for a serious problem and little more.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0340283 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in