Google Files Patent for an Assistant That Turns on Your TV When You Sit Down
Google is describing an assistant that glances at your living room through a camera, notices you sat down, and turns on the TV without being asked.
What Google's camera-aware assistant actually does
Every time you ask a voice assistant for something vague, like a dinner idea, it has to guess what you actually have on hand. It can't see your kitchen, so you end up answering follow-up questions or scrolling past recipes you can't make.
Google's patent application describes an assistant that can look before it answers. A camera in your fridge or living room sends pictures to an AI that describes what it sees, such as chicken on the shelf. That description gets folded into your request, so you get a chicken recipe instead of a generic list.
The claims go a step further. The system can come up with its own questions based on your day, check camera images for the answer, and then switch on a device, like your TV when you sit on the sofa. The filing says images would be used with your permission.
… generating an implied natural language input automatically based on the current context; processing the implied natural language input using one or more large language models (LLMs) to generate a natural language output indicative of the current context …
Translation: The system figures out what you want without you saying anything by looking at your surroundings.
How Google's two AIs pass notes to run your TV
The main claim chains two kinds of AI together. A large language model (LLM, the text-writing AI behind chatbots) and a vision language model (VLM, an AI that answers questions about pictures) pass notes to each other in plain text.
Here is the sequence the claim spells out:
- The system works out your current context. Follow-on claims say this can draw on your calendar, your location or your profile.
- It writes its own "implied" question from that context, with no typing from you.
- The LLM turns that into a short description, which is bundled with camera images into a prompt for the VLM.
- The VLM reports the state of the room, such as "the TV is on the wall and the sofa is white."
- Terms from that report go into a fresh LLM prompt, and the answer can be a command that operates a device.
The description's main example: you say, "Turn on this TV whenever I am sitting on the sofa." The LLM asks itself "Where is the TV?" and "What color is the sofa?" and the VLM answers from camera images. Later, the system keeps checking images, periodically or continuously, and when it sees a person sit down, it tells the TV to turn on.
Other claims say the system can match a spotted object against a registered device in your "coordinated ecosystem" (the gadgets tied to your account) to confirm it can be controlled. The description also suggests a slimmed-down VLM could run on your own device, so only text descriptions, not raw photos, go to the cloud.
Implementations relate to leveraging large language model(s) (LLMs) and vision language model(s) (VLMs) to facilitate human-to-computer dialogs.
Translation: This technology helps you talk more naturally with your smart devices using AI image and text analysis.
Why a home assistant with eyes raises the stakes
If this ever shipped, you could stop spelling out details to your assistant. Vague requests like "what should I cook tonight" would be answered from what a camera sees, and routines like "turn on the TV when I sit down" would run on their own, with no routine for you to wire up by hand.
The same camera that makes this handy also watches your home. The filing says images would be used with your permission, and it describes running a trimmed-down picture AI on the device so photos stay local. Whether people will trust a camera that keeps an eye on their living room is a bigger question than the patent answers.
Google's 52nd filing we've tracked since May in our AI agents that act for you watchlist adds to a pattern that includes an AI news analyst and a plain-English task planner.
Claim 1 is longer and more specific than the title suggests. The title promises chats that mix pictures and words. Claim 1 asks for more: the system invents its own question from your context, turns it into a description, hands that plus camera images to a picture-reading AI, and then uses the answer to operate a device.
Every step is listed, so the claim covers one particular chain of handoffs that ends in device control. A feature that only answers questions about a photo, with no device action at the end, does not match what claim 1 spells out. The wording "one or more devices" is wide, though, and covers any gadget an assistant could command.
The follow-on claims narrow things with calendar or location context, a person on a piece of furniture, and a registered smart device. This is still an application, so the patent office will decide how much of that breadth survives.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0320311 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in