IBM Patents a System for Scoring AI Bias Toward Protected Groups
Every large language model trained on internet text inherits the prejudices baked into that text. IBM's new patent describes a pipeline that hunts those prejudices down group by group, labels them, and rebalances the training data before the bias gets frozen into the model.
How IBM's bias-detection pipeline actually works
You're a developer at a bank, and your team has just trained a text-generation tool on years of internal documents. Someone asks: does this model talk differently about men than women, or about one religion versus another? Right now, answering that question takes a lot of manual work.
IBM's patented approach tries to automate that audit. The system starts with a master list of protected groups (race, gender, religion, age, and so on), each paired with keywords that signal when that group is being discussed. It scans a dataset, finds every sentence touching one of those groups, and then scores each sentence: does the text treat this group positively, negatively, or neutrally? The result is a labeled dataset that shows exactly where the imbalances are.
Once you know where the skew lives, the system tries to fix it by rebalancing how often positive, negative, and neutral examples appear for each group. The goal is a training set that doesn't systematically favor one group over another before any model ever sees it.
… executing a regard classification on each instance of the keywords to classify a regard towards a corresponding protected attribute, wherein the regard classification is one of a positive regard, a negative regard, or a neutral regard …
Translation: The system analyzes text to determine if the tone used toward a specific group is positive, negative, or neutral.
How regard scores get assigned and balanced
The patent describes a four-step pipeline for finding and correcting bias in datasets used to train AI models.
- Taxonomy of protected attributes: The system maintains a structured list of socially sensitive categories (gender, race, religion, disability, and similar groups), each with associated keywords. Think of it as a glossary that tells the system which words to watch for.
- Keyword matching: The pipeline scans a source dataset and flags every sentence or passage that contains one of those keywords, creating a collection of candidate instances for review.
- Regard classification: Each flagged instance is run through a classifier that judges the regard of the text toward the identified group. Regard is defined as one of three buckets: positive, negative, or neutral. This is similar to sentiment analysis (rating text as favorable or unfavorable), but it is anchored specifically to how a social group is portrayed rather than to general tone.
- Bias balancing: Using the labeled data as a reference, the system analyzes a second dataset and adjusts the distribution of positive, negative, and neutral examples for each group. The goal is parity: no single group should be disproportionately associated with negative language in the final training set.
The patent emphasizes that this process works across a taxonomy (a hierarchical, organized list) rather than ad hoc word lists, which is meant to make the coverage more consistent and auditable.
The mechanisms execute a bias analysis and balancing of regard classification distributions of a second dataset to mitigate undesirable biases based on the regard annotated dataset.
Translation: The software uses its findings to adjust data and reduce unfair treatment of protected groups in other systems.
What this means for AI fairness in enterprise software
AI fairness regulations are moving fast in both the EU and the US, and enterprises building text-based tools for hiring, lending, or customer service face real legal pressure to show they have audited their training data. A system that automates the audit and the fix, rather than relying on human reviewers reading millions of sentences, is a practical response to that pressure.
IBM sells heavily into regulated industries like banking, insurance, and healthcare, which are exactly the sectors where a documented bias-mitigation trail has the most value. The pipeline described here is software-only, meaning it could in principle be layered on top of existing data-processing workflows without new hardware. IBM's Watson-era AI tooling already included fairness products, so this fits a line of work the company has been building for years. Bias mitigation in enterprise AI is one of the more active areas covered across the newest Big Tech patents, and IBM's structured, keyword-driven approach reflects a bet that auditable, explainable methods will hold up better under regulatory scrutiny than black-box alternatives.
The ship path for this patent is short by AI-tooling standards. It is pure software, it does not require a new model architecture, and it slots naturally into a data-preparation stage that already exists in enterprise ML pipelines. The main open question is classification quality: a regard classifier is only as good as its training data, and the patent does not detail how that classifier was built or validated. If IBM can show the regard scores are reliable across languages and cultural contexts, the four-step pipeline here is close to a product feature.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0236562 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →