Microsoft Patents Live Camera Streaming to a Conversational Bot Without Draining Your Battery
Microsoft has filed a patent for a system that lets an AI assistant see what your camera sees while you ask it questions, and it's specifically designed to do that without burning through your data or battery in the process.
What Microsoft's camera-plus-AI assistant actually does
Right now, if you want an AI assistant to help you with something it needs to see, you typically send it a photo or a video clip. That's a clunky handoff: you stop, capture, upload, wait. Microsoft's patent describes a different approach, one where the AI gets a continuous stream of camera frames while you're still talking.
The system doesn't just blast full-resolution video at the AI, though. It cuts down the image quality when it can, and, more usefully, it only sends the parts of the frame that actually changed since the last snapshot. If you're holding a document steady, most of the image stays the same, so there's nothing new to transmit.
The result is that you could hold your phone over a broken appliance, ask "what's wrong with this thing?" and the AI would be watching the whole time you speak, without your phone getting hot or your data plan taking a hit.
… sending a content-bearing message to an agent interface that includes the instance of image data and the instance of query data; receiving a reply from the automated agent via the agent interface, wherein the reply is generated by the automated agent using a language model based on the instance of image data and the instance of query data …
Translation: The app sends pictures and questions together to an AI language model to get a smart reply.
How the client trims image data before sending it
The patent describes a client system (your phone, laptop, or similar device) that simultaneously captures audio or text input from you and image frames from a camera, then bundles both into messages sent to an agent interface, a middleware layer sitting between your device and the underlying AI language model.
Three bandwidth-saving mechanisms are called out specifically:
- Resolution reduction: the client shrinks images before sending them, so the AI gets a smaller file to process.
- Differential forwarding: only the portions of a frame that differ from the previous frame are transmitted (think of it like how video compression works: skip what hasn't moved).
- Prompt injection: the agent interface can insert extra context or instructions into the conversation automatically, either from a fixed list or dynamically, without the user having to type any of it.
The AI's reply is generated by a language model (the core prediction engine, similar to what powers ChatGPT) that receives both the image data and the spoken or typed question together. The whole loop, capture, send, reply, repeats continuously while the question is being asked, so the AI always has a fresh view of whatever you're pointing the camera at.
… the reduction in resolution of the instances of image data by the client system, the selective forwarding of only updated parts of the instances of image data that to the agent interface …
Translation: The system saves battery by shrinking image quality and only sending parts of the video that change.
What this means for AI assistants on phones and laptops
For everyday users, the pitch is that AI assistants that can see through your camera become much more practical on a real device with real bandwidth limits. A tourist asking an AI to identify a painting, a technician troubleshooting a server rack, a shopper comparing labels in a store: all of those involve pointing a camera at something and talking. Today those scenarios chew through data and drain batteries fast. A system that transmits only what changes makes that kind of interaction something you'd actually use regularly.
From a competitive angle, Microsoft's interest in multimodal AI assistants is obvious from products like Copilot, and this patent fills in a practical gap: the infrastructure to make camera-aware AI cheap enough to run on consumer hardware without a constant Wi-Fi connection.
Microsoft files its ninth application we've tracked since August, building on work like blocking bad data requests and a self-adjusting training system in our on-device AI privacy watchlist.
Claim 1 covers the basic loop of a device repeatedly capturing images while someone speaks a question, then bundling those images with the audio and sending them to an AI agent. That description fits any "look at this and answer me" assistant built around a camera and a voice prompt, which makes the claim broad enough to reach a wide range of products, not just one particular implementation.
Because the bandwidth-saving and resolution-scaling features live in dependent claims, a product that skips those optimizations entirely could still fall inside the core claim. That matters: the patent stakes out the interaction model itself, not a clever twist on it.
If granted as written, any similar camera-plus-voice AI loop would need to reckon with this claim, and that is a meaningful position to hold as that interaction model moves from novelty to everyday product feature.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
15 drawing sheets from US 2026/0289154 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in