Get the newsletter
IBM · AI / ML · Filed May 19, 2025 · USPTO published Nov 19, 2026 · This week's release

IBM Files Patent for Checking AI-Written Code Without Ever Running It

IBM's patent application describes a way to catch a bad AI-written command without ever running it. In one example from the filing, an AI asked to remove a user from a group wrote a command that added the user instead.

An evaluation tool catches errors in computer code, generates an explanation of what the script does, and grades its accuracy. Drawing from patent filing US 2026/0367884 A1.
An evaluation tool catches errors in computer code, generates an explanation of what the script does, and grades its accuracy.All 14 drawings ↓
Publication number
US 2026/0367884 A1(US20260367884A1)
Applicant
International Business Machines Corporation
Filed · Published
May 19, 2025 · Nov 19, 2026
Inventors
Brad Lee BLANCETT, Pooja AGGARWAL, Oishik CHATTERJEE, Ting DAI, Prateeti MOHAPATRA, Arthur DE MAGALHAES, Chad HOLLIDAY, Steven Paul BARBIERI, Eugen POSTEA
Status when we published
Waiting for an examiner
Document
20 claims
US classification
717/104

What IBM's code-checking patent actually does

Imagine you ask an AI assistant to remove a user named test from a group on your company's server, and it hands back a command that does the opposite: it adds the user. Paste that in without reading it closely and you've made a mess.

IBM's patent describes a checker that catches this kind of slip. Instead of running the command to see what happens, which can be risky on a real server, a second AI reads the code, describes in plain words what it does, and compares that description to what you asked for. When the two don't match, it explains why.

That explanation becomes feedback, and the feedback is used to rewrite the original instructions given to the AI, so the next attempt is more likely to get it right.

From the filing · CLAIM 1
… determining, using the machine learning component, a functional correctness of the generated code by performing a semantic validation operation without executing the generated code or comparing the generated code to a ground truth reference; …

Translation: The system checks if the code works by analyzing its meaning instead of running it or checking an answer key.

How the AI grades code it never runs

The system starts with a task described in plain language, such as fetch a list of top memory processes. An AI model (a large language model, or LLM, the software behind chatbots) writes the code. A machine learning component then works out what that code actually does and describes it.

Next comes the step claim 1 centers on, called semantic validation (judging whether code does what it is meant to do, rather than just whether it is well formed). It happens without running the code and without comparing it to a known right answer. The filing describes gathering similarity evidence and difference evidence between the request and the code, then ranking it. One ranking method counts how often each point appears across repeated analyses. Another predicts sample output and checks it against what the evidence implies.

The findings become a reasoning explanation, which feeds a feedback step. Feedback can use retrieval-augmented generation (RAG, where the AI is handed details about the error and possible fixes along with the question). The result is a refined prompt for the next round of code.

The filing also describes splitting big tasks into small steps, dropping steps that can't be automated, and pulling in missing system details. It focuses on system administration and shell scripts.

From the filing · THE ABSTRACT
A refined prompt is generated by refining a prompt using the generated feedback to improve subsequent code generation by the LLM.

Translation: It uses the lessons learned from checking the code to write better instructions for the AI next time.

Why this matters for AI-written server scripts

If you ever rely on AI to write commands for a computer you care about, the big worry is a confident wrong answer. The filing says testing generated code usually means running it in a carefully set up test environment, which is slow and costly, and that running code you don't trust carries security risks of its own. Checking by reading avoids both.

The filing aims at system administration and shell scripts, where a bad command can disrupt a server. It also says the checker improves over time: each mistake feeds back into the instructions the AI sees next, which could cut the manual review that eats up the time AI coding tools are meant to save.

That makes this IBM's 83rd filing we've tracked since May in the AI safety guardrails race, following its work on auto-patching chatbot attacks and stress-testing bots with rewritten prompts.

Source: US 2026/0367884 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 14 sheets

14 drawing sheets from US 2026/0367884 A1 · click any drawing to enlarge

How the system coordinates AI models to inspect code, drawing sheet 1 from US 2026/0367884 A1
Sheet 1 How the system coordinates AI models to inspect code
Deciding whether to find existing code or write new scripts, drawing sheet 2 from US 2026/0367884 A1
Sheet 2 Deciding whether to find existing code or write new scripts
Breaking down complex user tasks into automated steps, drawing sheet 3 from US 2026/0367884 A1
Sheet 3 Breaking down complex user tasks into automated steps
Generating code, fixing errors, and explaining what it does, drawing sheet 4 from US 2026/0367884 A1
Sheet 4 Generating code, fixing errors, and explaining what it does
Checking whether generated code matches the user's intent, drawing sheet 5 from US 2026/0367884 A1
Sheet 5 Checking whether generated code matches the user's intent
Using human reactions to improve future prompt instructions, drawing sheet 6 from US 2026/0367884 A1
Sheet 6 Using human reactions to improve future prompt instructions
Converting an open-files warning into actionable automation steps, drawing sheet 7 from US 2026/0367884 A1
Sheet 7 Converting an open-files warning into actionable automation steps
Catching script errors and confirming meaning without running them, drawing sheet 8 from US 2026/0367884 A1
Sheet 8 Catching script errors and confirming meaning without running them
Show the other 6 sheets
Human feedback helps update the prompt and code, drawing sheet 9 from US 2026/0367884 A1
Sheet 9 Human feedback helps update the prompt and code
An AI catches errors and corrects the code, drawing sheet 10 from US 2026/0367884 A1
Sheet 10 An AI catches errors and corrects the code
Test cases show how the AI spots flawed code, drawing sheet 11 from US 2026/0367884 A1
Sheet 11 Test cases show how the AI spots flawed code
Cloud servers and devices running the code checker, drawing sheet 12 from US 2026/0367884 A1
Sheet 12 Cloud servers and devices running the code checker
Internal parts of a device evaluating code, drawing sheet 13 from US 2026/0367884 A1
Sheet 13 Internal parts of a device evaluating code
Steps for validating AI code without executing it, drawing sheet 14 from US 2026/0367884 A1
Sheet 14 Steps for validating AI code without executing it
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.