Samsung Patents a System for Setting Device Automations With Voice and Gestures
Imagine pointing at your TV and saying 'turn this off when I leave the room' — and having your phone just figure out what you meant and make it happen. That's roughly what Samsung is trying to patent here.
What Samsung's voice-and-gesture automation actually does
Think about how you currently set up automations on your phone — navigating menus, picking triggers, choosing actions, saving rules. It's fiddly and most people never bother. Samsung's patent describes a much more natural approach: you just say what you want, and optionally show it too.
For example, you might say 'play music when I get home' while gesturing toward your speaker. The device interprets both your words and your physical behavior together to understand what condition should trigger what action. It then figures out on its own which sensors or data sources it needs to watch — your GPS, your front camera, a connected smart device — to actually carry that out.
The result is a system that turns casual, multimodal instructions into real, running automations — without you ever touching a settings screen.
How Samsung maps spoken rules to device detection resources
The patent describes a pipeline that combines voice information (what you say) and image information (what you physically do, captured via camera) to define an if-this-then-that style automation rule.
The system breaks your input into two parts: a condition (the trigger — e.g., 'when I leave the room') and an action (what happens — e.g., 'turn off the lights'). It then translates those natural-language and gestural inputs into a formal event to detect and a function to execute.
Once the rule is defined, the system automatically identifies the right detection resources — these are the sensors, APIs, or data streams (like motion sensors, location services, or connected device states) the phone should monitor to know when the condition is met. This is notable: the user doesn't have to specify how to detect the trigger, only what the trigger is.
When a detection resource reports that the event condition has been satisfied, the stored function fires automatically. The independent claim focuses primarily on the voice-only path, while the broader described approach also incorporates gesture and behavior recognition as a parallel input channel.
What this means for Samsung's smart home and Bixby strategy
Right now, voice assistants like Bixby are mostly reactive — you ask, they answer. This patent points toward a more proactive model where a single natural utterance (plus a gesture) can wire up a persistent, running automation. That's a meaningful step toward making smart home and device automation genuinely accessible to non-technical users.
For Samsung's broader ecosystem — Galaxy phones, SmartThings, Galaxy Home speakers — a system like this could serve as the glue layer that makes cross-device rules easy to set. If this makes it into a shipping product, you could configure complex smart home behavior in seconds just by describing it out loud.
This is a legitimately useful UX idea that addresses a real friction point: automation setup is too hard for most people and voice assistants are too stateless. The multimodal angle (voice plus gesture) is the interesting differentiator — though the independent claim quietly strips the gesture component down to voice-only, which is worth noting. Whether Samsung can execute on this in a way that's reliable enough to trust is the real question.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.