GM Files Patent for Automated Testing of Car Touchscreen Menus
Before a new version of your car's touchscreen reaches you, someone has to check every button on it. GM's patent describes software that finds each tap and swipe zone on a screenshot and tests them automatically.

- Publication number
- US 2026/0368873 A1(US20260368873A1)
- Applicant
- GM GLOBAL TECHNOLOGY OPERATIONS LLC
- Filed · Published
- May 13, 2025 · Nov 19, 2026
- Inventors
- Vishranthi Nairi
- Status when we published
- Examiner raised objections; the company can respond
- Document
- 20 claims
- US classification
- 382/103
- Examiner
- BLACKSTEN, SYDNEY LYNN (Art Unit 2674)
What GM's touchscreen-testing tool actually does
Every time a carmaker updates its touchscreen software, a tester has to take screenshots, crop out each button, write down where it sits, and then check that it still works. Then they do it all over again for the next version.
GM's filing describes a tool that skips most of that grunt work. Feed it a picture of a screen, such as the climate controls, and it spots every tap and swipe area on its own. It saves a small image of each one, along with its exact position and size.
Then it uses that map to pretend to be you, tapping and swiping through the interface to see whether everything responds. If something fails, the screen design gets fixed. When a new software version arrives, the tool only has to deal with buttons that changed.
… identifying input regions on the image of the user interface, the input regions to receive a user input of a user input type; for each of the input regions of the image of the user interface, generating an input region image and location information …
Translation: The system scans the screen to find every button and menu option that a user can touch.
How the tool finds and tests each button
The patent covers a testing pipeline for in-car screens such as infotainment, climate control and navigation. It starts with an image of the interface, which can be a live feed from the display or a stored screenshot.
A detection and extraction engine then finds the input regions, meaning the spots meant to receive a touch. The filing says it can use a neural network (software trained on examples to recognize patterns) to do this. It also describes a manual option where a person drags a box over each region with a mouse. In the example figure, a climate screen has 22 regions, each saved as its own image file.
For each region, the tool records location data:
- Tap regions: a starting coordinate, plus a width and a height.
- Swipe regions: a start point, an end point, a width, a height, and midpoint values.
A testing engine uses that data to simulate many user inputs and check how the interface responds. Results can lead to changes in the layout, the input areas, or the software itself.
Everything goes into a database. When a new version of the screen shows up, the tool compares it with the old one, spots new or changed regions, and updates only those.
… performing a validation test on the user interface of the system of the vehicle using the input region image and the location information for each of the input regions.
Translation: It runs automated checks on those touch points to make sure everything works correctly.
Why GM's screen tester matters for software updates
If this works as described, it would mostly help GM's own engineers and testers. Fewer manual screenshots and spreadsheets could mean software updates get checked faster, and with fewer mistakes from tired humans. For you, that could show up as fewer touchscreen bugs, such as a button that stops responding after an update.
The filing is a software tool, not a car feature. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. Nothing here says whether or when GM would use it in practice.
GM's third patent application in our GM coverage since November follows one on echo cancellation for calls and one on heating a cold EV battery.
Source: US 2026/0368873 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time GM patent news breaks.
We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.
The drawings · 4 sheets
4 drawing sheets from US 2026/0368873 A1 · click any drawing to enlarge

No comments yet