Samsung Patents Voice Control for Apps That Were Never Built for It
Plenty of apps never added voice control, which leaves your voice assistant with nothing to grab. Samsung's patent has the phone read what is on the screen and press the right button itself.
What Samsung's screen-reading voice control does
A woman stands at the stove with wet hands and tells her phone to play a song. The music app on the screen was never built to listen, so nothing happens and she has to wipe her hands and tap.
Samsung's patent goes after that gap. Instead of waiting for every app maker to add voice support, the phone looks at what is already on the screen: the song titles, the button labels, the names in a chat list. It turns your words into a command (play) and a target (the song name), then checks whether the screen has a matching button.
If it finds one, it presses it for you. If it does not, it moves to a different screen that can get the job done, such as opening the right chat before sending your message. If the song is already playing, it leaves the screen alone.
… identify, based on a comparison between the first information and the second information, whether a target object corresponding to the user voice is present from among the at least one object …
Translation: The software checks if what you said matches anything currently shown on the screen.
How the phone matches your words to a button
When you speak while an app screen is showing, a speech recognition model (software that turns audio into text) writes out your words. A second step pulls out two things: the command and its target. Say "turn on LOVE DIVING" in a music app and the command becomes PLAY, with the song title as the target.
At the same time, the device reads data describing the current screen and lists what is on it. For each object, it notes the text shown and, for icons, the function they stand for, such as a play button. Then it compares the two lists. A match can be exact, partial, or similar, judged by fixed rules or a trained model.
One twist: the screen is split into a constant area and a variable area. In the filing's chat example, you are in a chat with SAM, SIHO, while another name, CHIL, SIHO, shows up inside a message. If you ask to message CHIL, SIHO, the phone ignores the changing area, finds no match, and opens CHIL's chat on a third screen instead.
The device also stores a map for each app: screens as nodes and the actions linking them (tapping, opening a text field) as edges. It compares a numeric fingerprint of the current screen (an embedding vector) with stored ones to work out where it is, then follows connected nodes to the right next screen.
… obtain first information including information on a command and information on an execution target of the command included in the text information …
Translation: The system figures out what action you want to take and what you want to do it to.
Why apps without voice support could benefit
Voice assistants usually work well only in apps whose makers built voice support in. Everything else makes you tap. A phone that can read a screen and press the right button itself could close that gap, which would matter most when your hands are busy, wet, or full.
This is a patent application, so nothing here says Samsung will ship it. The filing describes a general method for any device with a microphone and a display, and it says the speech model can run on the device or on a server. If it ever reached products, the test would be simple: does your voice work in the app you use most?
Samsung's 17th filing we've tracked since July in our AI agents acting for you watch, following patents like the hold-call assistant and weather-aware cooling, adds another layer to its vision of software that handles tasks on your behalf.
Samsung's approach here trades certainty for reach. Instead of requiring every app to be built with voice commands in mind, the phone reads the labels already on screen and makes its best guess about what you meant. A guess can be wrong, and in a messaging app, a wrong guess sends your message to the wrong person.
The filing does account for this. The phone deliberately ignores parts of the screen that change quickly, like a scrolling chat history, so it doesn't confuse a contact's name with a passing message. That caution adds a small extra step sometimes, but it is a reasonable price for not texting a stranger by accident.
The quieter cost is one the filing only hints at: app makers still have to label their buttons clearly, or the whole system stumbles. Sloppy labels produce sloppy results. That dependency on developers doing their part is real, but the overall trade reads as worth it.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Samsung patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0349652 A1 · click any drawing to enlarge
Show the other 1 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in