Samsung Patents a Way to Let You Ask Questions About a Video While It's Playing
You're watching a cooking show and wonder what that ingredient is, or you're mid-movie and want to know who that actor is. Samsung's new patent describes a system that can answer those questions in real time, without pausing the video.
What Samsung's live video AI assistant actually does
You're watching a travel documentary and a stunning building fills the screen. You have no idea what it is or where it is, and by the time you've typed a search, the shot has moved on. Samsung's patent describes a system designed to fix exactly that moment.
As a video plays on your device, the system continuously scans the screen and logs every recognizable object it spots, keeping them in a short-term memory buffer. When you type or speak a question, it checks that buffer for the most relevant thing it's seen, then sends both your question and that visual snapshot to an AI language model. The AI responds with an answer based on what was actually on screen, not just your words.
The key detail is timing: the system tosses those visual snapshots after a set window of time. It's not building a permanent record of everything you watch. It's more like a short-term visual scratchpad that only sticks around long enough to answer your question.
detecting visual objects in a video concurrently with playing the video on a screen of a device; storing the visual objects within a physical memory of the device and discarding the visual objects from the physical memory after a predetermined window of time …
Translation: The device identifies items on screen while you watch, but it only keeps them in temporary storage for a short period.
How Samsung detects objects and routes your question to an AI
The system works in three overlapping stages running at the same time video plays.
Object detection: A computer-vision layer scans each frame of the video as it renders on screen, identifying discrete visual objects (a building, a food item, a person's face, a car model). These detected objects are written into the device's physical memory, meaning they're stored in fast-access RAM rather than on disk.
Short-term memory management: The system enforces a predetermined window of time, a rolling expiration timer, after which stored objects are discarded. This keeps memory use in check and avoids accumulating a long-term log of viewing history on the device.
Query matching and AI routing: When the user submits a query (typed or spoken), the system searches its current in-memory object list for the closest match. Once it finds the best match, it packages that visual object together with the user's original question and sends both to a large language model (LLM) system (an AI that processes language and context, similar to what powers chatbots). The LLM then generates a response based on that combination of image context and user intent, and the answer is returned to the user.
The claim is device-side: detection and memory management happen on the playback device itself, while the heavy AI reasoning is offloaded to the LLM system.
In response to matching a selected visual object from the physical memory with the user query, the selected visual object is submitted with the user query to a large language model system. A result from the large language model system is provided to the user.
Translation: When you ask a question about an object, the system sends that image and your text to an AI to get an answer for you.
What this means for how you interact with your TV or phone
For the person watching, this closes a gap that has existed since video went digital. Pausing, screenshotting, opening a separate search app, and then trying to describe what you just saw is a clunky workaround that most people skip entirely. A system that keeps a live visual buffer means the device already knows what you were looking at when you ask, so your question can be as vague as "what's that?"
The time-limited memory window is the part of this patent that matters most for privacy-conscious users. Because visual snapshots expire automatically, there is no persistent log of your viewing history sitting on the device. Samsung has been expanding its on-device AI features across televisions, phones, and tablets, and the latest Big Tech patents in the video-AI space suggest real-time content understanding is quickly becoming a battleground for which platform can answer your viewing questions fastest.
The reader-facing payoff here is specific and immediate: you get an answer about what's on screen without breaking your viewing experience or opening a second app. The time-limited memory detail shows Samsung has thought about the obvious concern, that a device silently cataloging everything you watch would be deeply uncomfortable, and built an expiration mechanism directly into the core claim. Whether that window is five seconds or five minutes matters enormously in practice, and that spec is conspicuously absent from the filing. That gap will determine whether this feels like a useful feature or a privacy afterthought when it ships.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
4 drawing sheets from US 2026/0236530 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →