Get the newsletter
Google · AI / ML · Filed Apr 28, 2026 · USPTO published Nov 19, 2026 · This week's release

Google Patents a Video-Call-Style AI Chat That Sees Your Screen or Camera

Google's patent application turns asking an AI about what you see into something that works like a video call. Show it your camera or your screen, talk, and it answers out loud.

A phone camera views a folded paper airplane while an AI assistant explains the next step and shows related web results. Drawing from patent filing US 2026/0367867 A1.
A phone camera views a folded paper airplane while an AI assistant explains the next step and shows related web results.All 24 drawings ↓
Publication number
US 2026/0367867 A1(US20260367867A1)
Applicant
Google LLC
Filed · Published
Apr 28, 2026 · Nov 19, 2026
Inventors
Jessica Lee, Clement Dickinson Wright, Harshit Kharbanda, Wewage Anuda Nethum Dep Weerasinghe, Kai Yu, Louis Wang, Stacy Lou Chiou, Sarah Fay Smith, Christopher James Kelley, David Kao-Chun Wu, Christophe Patrice Fondacci, Liza Sol Wah Ma, Olawale Abiri, Kimiya Hojjat
Status when we published
Waiting for an examiner
Document
20 claims
US classification
715/709
Parent application
Claims priority from a provisional application 63806455 (filed May 15, 2025)

What Google's video-call AI chat actually does

Searching for something you're looking at usually means turning it into words first, and that's harder than it sounds. Maybe you can't say what an object in front of you is, or the words you pick bring back the wrong results. Google's patent application describes a way around that: you start a chat that works like a video call with an AI.

You can point your camera at something, or share your phone screen, and just ask out loud. The AI watches what you're showing it, listens to your question, looks things up on the web, and talks back. Little cards with source links pop up on screen so you can check the answer.

The filing also covers a few handy extras. You can start with a voice-only call and add video later, and the AI can read the whole webpage you're on, not just the part you can see. Afterward you can scroll through a transcript of the session.

From the filing · CLAIM 1
… obtaining, by the computing system, an image stream from the user computing device, wherein the image stream comprises a plurality of image frames; obtaining, by the computing system, an audio stream from the user computing device, wherein the audio stream comprises audio data obtained via one or more audio sensors of the user computing device; …

Translation: The system collects what your camera and microphone pick up from your device.

How the AI watches, listens, and searches for you

The claims cover a chat that takes two live feeds from your device: an audio stream (your voice, picked up by the microphone) and an image stream (frames from your camera or from a capture of your screen). A generative model (an AI that writes new text rather than just retrieving it) reads both and writes its own search query. A search runs, and the model combines the results with the query to write a reply that plays through your speaker.

The filing describes three ways in:

  • A voice-only call.
  • A viewfinder mode that uses your camera.
  • A screensharing mode that captures what's on your display through an overlay.

To keep things fast, the system can sample frames at a variable rate, using pixel diffing (checking how much the edges of the picture change from frame to frame) so a still scene costs less to process. The claims mention between three and five minutes of frames, and the description says older visuals can be shrunk into short text summaries. During screensharing, the capture layer sits below the on-screen indicator, so the AI's own indicator never blocks its view.

It can also work out which webpage you're on, then fetch the full page or its source code, and it can save a transcript with the search results and relevant frames.

From the filing · THE ABSTRACT
Systems and methods for providing a multimodal video chat interface can obtain data descriptive of an audio stream and an image stream, can process the audio stream and image stream to generate a model-generated query, can determine search results responsive to the model-generated query, and can generate a model-generated response based on the search results.

Translation: It takes your video and voice input, turns them into a search, and builds a smart reply from the results.

Why Google wants search to feel like a call

The filing's own examples are everyday ones: troubleshooting an appliance by showing it to your camera, comparing prices across browser tabs, or getting a long on-screen document summarized. You skip typing a search and just talk about what's in front of you.

The design treats the whole thing as a phone call with an assistant. You start with voice, can upgrade to video, and can switch between camera and screen mid-chat. The filing says it could run in a virtual assistant app on a phone or alongside a smart wearable. Whether it ever ships is unknown, since this is a patent application.

This is the 64th Google filing we've tracked since May on AI agents acting for you, joining earlier work on earbud voice filtering and a TV app-learning assistant.

Source: US 2026/0367867 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 24 sheets

24 drawing sheets from US 2026/0367867 A1 · click any drawing to enlarge

An AI model turns live audio and video into answers, drawing sheet 1 from US 2026/0367867 A1
Sheet 1 An AI model turns live audio and video into answers
Adding camera feeds and screen capture to a voice call, drawing sheet 2 from US 2026/0367867 A1
Sheet 2 Adding camera feeds and screen capture to a voice call
How the system finds web results during live chat, drawing sheet 3 from US 2026/0367867 A1
Sheet 3 How the system finds web results during live chat
Starting out as an audio call with web search cards, drawing sheet 4 from US 2026/0367867 A1
Sheet 4 Starting out as an audio call with web search cards
Using your phone camera to share your workspace, drawing sheet 5 from US 2026/0367867 A1
Sheet 5 Using your phone camera to share your workspace
Viewing the written history after you finish speaking, drawing sheet 6 from US 2026/0367867 A1
Sheet 6 Viewing the written history after you finish speaking
Asking questions while sharing an app on your screen, drawing sheet 7 from US 2026/0367867 A1
Sheet 7 Asking questions while sharing an app on your screen
Different ways to launch a live voice or video session, drawing sheet 8 from US 2026/0367867 A1
Sheet 8 Different ways to launch a live voice or video session
Show the other 16 sheets
Asking follow-up questions during a live video call, drawing sheet 9 from US 2026/0367867 A1
Sheet 9 Asking follow-up questions during a live video call
Switching from a video chat to web search results, drawing sheet 10 from US 2026/0367867 A1
Sheet 10 Switching from a video chat to web search results
How the AI watches your screen and answers questions, drawing sheet 11 from US 2026/0367867 A1
Sheet 11 How the AI watches your screen and answers questions
Using screenshots and your voice to find web results, drawing sheet 12 from US 2026/0367867 A1
Sheet 12 Using screenshots and your voice to find web results
Connected devices and servers running live audio and video, drawing sheet 13 from US 2026/0367867 A1
Sheet 13 Connected devices and servers running live audio and video
Programs that inspect camera feeds to build answers, drawing sheet 14 from US 2026/0367867 A1
Sheet 14 Programs that inspect camera feeds to build answers
Training the AI models using feedback on their generated responses, drawing sheet 15 from US 2026/0367867 A1
Sheet 15 Training the AI models using feedback on their generated responses
An AI model transforming user inputs into finished answers, drawing sheet 16 from US 2026/0367867 A1
Sheet 16 An AI model transforming user inputs into finished answers
An AI model predicting responses from a sequence of inputs, drawing sheet 17 from US 2026/0367867 A1
Sheet 17 An AI model predicting responses from a sequence of inputs
Combining text, audio, and images into an ordered queue, drawing sheet 18 from US 2026/0367867 A1
Sheet 18 Combining text, audio, and images into an ordered queue
Tools developers use to build and refine the AI, drawing sheet 19 from US 2026/0367867 A1
Sheet 19 Tools developers use to build and refine the AI
The staged training routine that sharpens AI performance, drawing sheet 20 from US 2026/0367867 A1
Sheet 20 The staged training routine that sharpens AI performance
Cloud servers answering requests with live user and web data, drawing sheet 21 from US 2026/0367867 A1
Sheet 21 Cloud servers answering requests with live user and web data
Connected devices swapping data with cloud servers, drawing sheet 22 from US 2026/0367867 A1
Sheet 22 Connected devices swapping data with cloud servers
Multiple phone apps each running dedicated AI tools, drawing sheet 23 from US 2026/0367867 A1
Sheet 23 Multiple phone apps each running dedicated AI tools
A shared system hub powering AI across every app, drawing sheet 24 from US 2026/0367867 A1
Sheet 24 A shared system hub powering AI across every app
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.