IBM Files Patent for an AI That Checks Other AI Against the Law
The EU AI Act runs 108 pages, and over 900,000 AI models are already out there. IBM's filing has one AI read the law and others grade the models.
What IBM's AI-law checker actually does
You run a company that wants to put an AI chatbot in front of patients, and your lawyer asks a simple question: does this thing follow the rules? Today the honest answer is a lot of reading. The EU AI Act alone runs 108 pages plus a 17-page annex, and new laws keep arriving.
IBM's filing describes software that does the reading for you. One AI model pulls the individual requirements out of a law. Then the system checks an AI model against each one, either by reading its model card (the public fact sheet a model's maker writes about it) or, when that is not enough, by quizzing the model directly.
A second AI model then grades the answers and returns a result, such as a simple yes or no.
chunking a model card of the evaluated machine learning model by a chunking component of the system outputting a set of chunks; transforming, by executing a specially purposed code within a retrieval augment generation component a chunk from the set of chunks based on a similarity measure between a vector of the chunk from the set of chunks and the requirement dataset …
Translation: It breaks down the target AI's documentation files and matches them against legal requirements using vector math.
How the model card and test prompts get graded
The system starts with an extractor model, a language model given custom prompts, that takes in a stream of regulations and outputs a list of requirements. Each requirement gets a classification. In the filing's example, 36 requirements come out of Annex IV of the EU AI Act.
That classification picks the route. If the model being checked has a model card, the requirement is static. If the model has refused to answer questions on the topic before, it is dynamic.
The static route works in three steps:
- The model card is chunked, meaning cut into short sections.
- An embedding model turns each chunk into a vector (a string of numbers that captures meaning). A retrieval step (RAG, which hands a model only the passages relevant to a question) ranks chunks by cosine similarity (a closeness score) against the requirement.
- The model being checked gets the top chunks and outputs a compliance metric, such as a yes backed by a quoted passage. A scoring model can turn that into a zero or one.
The dynamic route derives a set of test prompts from the requirement, for example a rule against using AI in criminal activity. The model answers them, and a compliance evaluator model (the filing calls it an auto-judge) outputs a result. That judge can be fine-tuned (retrained) on annotated examples and checked with accuracy and agreement scores.
Why AI makers may want an automated auditor
If you use an AI tool at work, somebody has to show it follows the law, and that gets slow when the rules differ by place. The EU, China, the US federal level and states such as Colorado all have their own. An automated checker could cut down the reading and be rerun when a rule or a model changes.
The filing points at healthcare, law and finance, where a misleading chatbot answer can do real harm. It also says the software could be offered as a hosted service or as part of a consulting engagement with a client. This is a patent application, so nothing in it says IBM has built or will sell such a product.
That makes this IBM's 70th filing we've tracked since May in the AI guardrails race, building on earlier applications like one on tracing text to its source model and one on picking the best AI model.
The final compliance verdict comes from another AI model grading the one under review. That is the central bet, and it is a real one: the grader can be wrong, and the filing knows it, so it monitors the grader's accuracy and retrains it when performance slips. A grader is only as reliable as the test cases it was checked against.
The other weak point is the source material. Compliance is partly measured against a document the model's own maker wrote, which reveals what was claimed, not how the model actually behaves when pushed. The test-prompt approach covers some of that gap, but no fixed set of questions can anticipate every situation a regulation is meant to address.
Still, the trade reads as worth it for the purpose it serves. A human lawyer starting from a flagged shortlist is faster and cheaper than one reading a hundred-page law cold. As a final legal verdict, this would be too thin, and the filing does not pretend otherwise.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time IBM patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0340343 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in