Get the newsletter
Ford · AI / ML · Filed May 14, 2025 · USPTO published Nov 19, 2026 · This week's release

Ford Files Patent for Finding Out Why Its Camera AI Mislabels Images

When a car's camera AI calls a photo one thing and a human expert says it's another, nobody can say why. Ford's new patent builds a tiebreaker image that sits exactly on the line between the two answers.

A computer scans a photo of a bird and breaks down its shapes to identify what kind of animal is in the picture. Drawing from patent filing US 2026/0369035 A1.
A computer scans a photo of a bird and breaks down its shapes to identify what kind of animal is in the picture.All 4 drawings ↓
Publication number
US 2026/0369035 A1(US20260369035A1)
Applicant
Ford Global Technologies, LLC
Filed · Published
May 14, 2025 · Nov 19, 2026
Inventors
Jian Hu, Gengxin Li, Ming Gao, Amit Ramesh Karjol, Steven Kraine, Ali Hassani, Yuming Niu
Status when we published
Waiting for an examiner
Document
20 claims
US classification
382/224
Examiner
ZHANG, LUCY PUYU (Art Unit 2676)

What Ford's image-explaining patent actually does

Ever had a computer insist on something that is plainly wrong, and then give you no clue why? Image-recognition software does this all the time. It looks at a photo, picks a label, and never explains itself. For a car that reads lane lines and traffic lights, that is a real headache for the engineers.

Ford's patent application describes a way to ask the software what tipped it. Engineers pick a photo the AI labeled one way, plus a similar photo it labeled another way. The system then makes a new image that sits right on the fence between the two labels, where the AI is almost evenly split.

By studying what had to change to reach that fence, the system pinpoints the few visual details that drove the original call. An engineer can then use that list to adjust how the AI is trained.

From the filing · CLAIM 1
… generate a counterfactual image that is on a boundary separating the first class and the second class …

Translation: Ford's system creates a hybrid image right on the dividing line between two different categories.

How Ford finds the pixels that tip the AI's call

The patent covers a computer that takes two images: a query image (one the classifier put in a first class) and a reference image (a similar one it put in a second class). It then generates a counterfactual image, meaning a "what if" version of the query photo, tuned to land exactly on the boundary between the two classes.

The image is built by optimizing an objective function (a scoring formula the computer tries to improve). The formula pushes the image toward the second class, keeps it close to the original photo, and limits changes to features that the query and reference images already share. A feature here is a small piece of information about a region of the image, like an edge or a corner.

From that boundary image, the system picks out features that changed a lot, then ranks them. Ranking works by generating more boundary images while holding each feature fixed and measuring how much that changes the result.

Finally, it works down the ranked list:

  • Change the top feature in the original photo and re-check the label.
  • Add the next feature, and re-check.
  • Stop as soon as the label flips to the second class.

The features changed before the flip form the dispositive features, the smallest set that explains the call. The filing says an operator can use them to adjust training.

From the filing · THE ABSTRACT
… determine a set of dispositive features responsible for the machine-learning classifier classifying the query image into the first class, based on the counterfactual image …

Translation: It pinpoints the exact visual details that tricked the AI into making its mistake.

Why Ford wants to see inside its camera AI

Ford's description says the technique can surface causes that a human expert would not suspect. It can also explain a classification when the expert believes the AI should have said something else. That matters for any camera system that has to read the road correctly, since the filing mentions a vehicle recognizing a lane line or a green light and then steering or holding speed.

For you as a driver, the benefit is indirect. A better explanation gives engineers a better way to fix mistakes before software ships. The filing is about debugging an AI's training, not a new feature you would see in the car. It does not say whether Ford uses this in any current product.

Source: US 2026/0369035 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Ford patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 4 sheets

4 drawing sheets from US 2026/0369035 A1 · click any drawing to enlarge

How the system inspects images to identify objects, drawing sheet 1 from US 2026/0369035 A1
Sheet 1 How the system inspects images to identify objects
Tracking test images across the border between two categories, drawing sheet 2 from US 2026/0369035 A1
Sheet 2 Tracking test images across the border between two categories
Finding which visual details trigger a wrong classification, drawing sheet 3 from US 2026/0369035 A1
Sheet 3 Finding which visual details trigger a wrong classification
Car parts that use the retrained vision software, drawing sheet 4 from US 2026/0369035 A1
Sheet 4 Car parts that use the retrained vision software
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.