Google Files Patent for a TV Assistant That Learns What Your Apps Can Do
Google's patent describes a TV assistant that does not need a hand-built command for every task. It reads a list of what each app can do, then picks the right app when you speak.
What Google's TV voice assistant actually does
Ever told your TV to "show me my photos" and heard back "Sorry, I don't know how to help with that"? That dead end happens when nobody programmed that exact request. This Google patent describes a way around it.
Instead of waiting for engineers to add each voice command, every app would publish a short list of its own abilities, such as "search" or "add to list." Your TV's assistant reads those lists. When you ask for your photos, it matches your words to a photo app and opens it. If two apps could do the job, it asks which one you want. It can even look at a screenshot of an app to figure out what you could do on that screen.
The assistant could also check your other devices, and an app store, for apps that would help, then suggest you download one.
accessing a capabilities application programming interface for interfacing with a capabilities repository; identifying at least one capability included in the capabilities repository for use in performing the action; and determining that an application associated with the at least one capability is installed on the computing device; …
Translation: The assistant checks a central database to figure out which installed apps can handle your request.
How the assistant reads each app's ability list
The patent describes an "agent," a software helper built on a large language model (an AI trained to understand ordinary speech) that runs on a smart TV. A separate voice assistant first turns your speech into text. The agent then works out how to carry out the request.
It does this with a few building blocks:
- Capabilities repository: a list of what each app can do. Apps declare abilities in a structured file, with example phrases and a deep link (a direct shortcut that opens a specific function inside the app).
- Screenshot reading: if an app has not published a list, the agent can analyze a screenshot of it. On a profile-selection screen, "select John's profile" would pick John.
- Context repository: a second list that tracks what is on screen, the device type, and what you said earlier. That lets "find this on the streaming app" know what "this" means.
An orchestrator (the planner) builds a response plan, and a router turns it into actions, such as opening an app's search with a show's name filled in. If something is missing, like a brightness percentage, or two apps fit, the agent asks you.
The lists can sync across your devices, and a central app-store list lets the agent suggest downloads. All three independent claims (method, storage medium, system) cover the same core steps: get text from speech, look up a matching capability, confirm the app is installed, and launch it.
… receiving, by an agent included on a computing device, input text as a sequence of words based on voice data that indicates an action for performing on the computing device.
Translation: The device listens to your voice command and converts it into text to figure out what you want to do.
Why TV voice commands could get less rigid
Voice control on TVs often fails because every command had to be built by hand, one feature at a time. This filing tries to flip that. Any app could announce its own skills, and the assistant would pick them up. For you, that could mean installing a photo app and having "show me my photos" simply work, in your own words.
The filing centers on TVs and remotes, but it also describes phones, tablets and laptops sharing the same ability lists. The examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments. The patent does not name a product, so treat it as a look at the problem Google is trying to solve.
Google's 62nd filing we've tracked in our AI agents acting for you watch since May builds on ideas like sorting and answering messages and tap-to-answer calls.
If this works as described, the change you would notice first is fewer dead ends. Today, saying something slightly off script to a TV often gets a shrug. Here, the assistant would match your own wording to something an app says it can do.
The follow-up questions matter just as much. Hearing "Which photo app do you mean?" or "What brightness percentage?" beats a silent failure, and you would likely notice that more than anything clever underneath.
The catch is that the payoff depends on app makers publishing good ability lists, and on you wanting to talk to your TV at all. For people who already use a voice remote, it is a sensible fix for a familiar annoyance. For everyone else, little changes.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
8 drawing sheets from US 2026/0351254 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in