Get the newsletter
Google · AI / ML · Filed Feb 4, 2026 · USPTO published Jun 18, 2026 · Jun 18 release

Google Patents a System That Tests New Phone Features Before They Reach Users

Before Google ships an AI update to millions of phones, it wants to run a controlled head-to-head race on a sample of real devices — and only promote the winner.

By · Updated · 3 min read

Google Patent: Testing AI Models on Real Devices Before Rollout, figure from US 2026/0169885 A1
Figure from the official USPTO publication.
Publication number
US 2026/0169885 A1(US20260169885A1)
Applicant
GOOGLE LLC
Filed · Published
Feb 4, 2026 · Jun 18, 2026
Inventors
Dragan Zivkovic, Harry Bleyan, Tamar Lucassen, Akash Agrawal, Françoise Beaufays
Status when we published
Waiting for an examiner
Document
20 claims
US classification
702/182
Parent application
Continuation of 17/717,822 (filed Apr 11, 2022)(17717822)

How Google picks which AI model lands on your phone

Imagine a sports team that never sends a player onto the field without first watching them practice against a real opponent. Google's patent describes something similar for AI software updates.

Right now, when a company wants to update the AI running on your phone — say, the model that powers autocomplete or voice recognition — it typically tests that update in a lab. This patent describes a different approach: pick a small group of real phones that share similar specs, run the new model on some of them and the old model on others, then measure which one actually performs better on those devices.

If the new model wins the head-to-head test — using real metrics like speed, memory use, and accuracy — Google can roll it out to the broader group of phones with the same specs. If it loses, the update gets held back or tweaked. Your phone only gets the update if it's been proven to handle it well.

How the device-grouping and head-to-head test works

The patent describes a server-side system that coordinates a structured A/B test across two groups of real client devices (phones, tablets, etc.).

  • Group selection by device profile: The system picks two subsets of devices that share the same hardware and software characteristics — same chip, same OS version, same memory tier. This keeps the comparison fair.
  • Parallel benchmarking: One group runs the existing ML model; the other runs the candidate replacement. Each device generates performance measures — latency (how fast it responds), memory consumption, CPU load, and accuracy metrics like precision and recall (how often it gets the right answer and how rarely it misses).
  • Winner selection and deployment: The system aggregates those measures, picks the better-performing model, and deploys it to both groups — and potentially to any other devices in the fleet that match the same hardware profile.

The patent also covers a feedback loop: if neither model clearly wins, the system can modify the candidate model and run the test again. The scope covers not just ML models but also non-ML software features and hardware-enabled functionality, making this a general-purpose quality gate for any on-device update.

What this means for on-device AI quality control

On-device AI is only getting more common — voice assistants, camera processing, keyboard prediction, and health sensors all run local models. The traditional way to evaluate those models is benchmarking in a controlled lab, which can miss real-world problems: a model that runs fine on a test rig might stutter on a three-year-old mid-range phone in someone's pocket.

This patent suggests Google wants a more systematic, data-driven gate between an AI update and your device. For Pixel phones and Android broadly, that could mean fewer cases where an update quietly degrades battery life or response speed on older hardware. It's essentially a quality-control layer that runs on the actual population of devices rather than simulated conditions.

Source: US 2026/0169885 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

Add Patentlyze on Google One click.
There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

More from Google

All 1,079 Google patents →

More in AI / ML

All 2,637 AI / ML patents →