Samsung Patents a Robot That Figures Out What You're Looking At

By Patentlyze Team · Updated Jun 5, 2026

Imagine telling a robot 'bring me that' while glancing at a bottle on the counter — and it actually knows what 'that' is. Samsung's new patent describes exactly that kind of gaze-aware robot control.

What Samsung's gaze-tracking robot actually does

You know how when you're talking to someone and you glance at an object, they usually understand what you mean? This patent is Samsung trying to give robots that same social awareness.

Here's how it plays out: a robot's camera watches you, figures out where you're standing and which direction your eyes are pointed, then cross-references that with a map of the room it already knows. An AI model takes all that location and gaze data and identifies the specific object you're probably looking at — the lamp, the cup, the door.

Once the robot knows what you're focused on, it can respond to a command like 'pick that up' or 'tell me about that' without you having to name the object or point with your hand. It bridges the gap between how people naturally communicate and how robots need explicit instructions to act.

How the robot maps your gaze onto its environment model

The patent describes a multi-step pipeline that converts raw camera footage into actionable intent. When you stand in front of the robot, its camera grabs an image of you and immediately extracts two things: your position relative to the robot and your gaze direction — essentially a vector pointing from your eyes outward into the room.

That relative information then gets translated into absolute coordinates on a pre-built map of the robot's operating environment. The robot knows where it is on that map at any given moment (via its own positioning data), so it can do the coordinate math to place you and your line of sight onto that same map.

Next, the gaze and position data get encoded onto a grid map — think of it like a top-down floor plan where each cell represents a small patch of floor or space. That grid becomes the input to an AI model (a trained neural network) that was specifically taught to associate gaze vectors on maps with physical objects in the environment.

The output is a confident identification of the object you're looking at. After that, when you issue a voice command or gesture, the robot already knows the target — so it can fetch, describe, or interact with that object without needing you to spell it out.

What this means for home and service robots

For home and service robots to feel genuinely useful rather than clunky, they need to understand human-native communication — and humans constantly use gaze as a pointing mechanism without even thinking about it. A robot that can't follow your eyes forces you to adapt to it, which slows everything down.

This patent positions Samsung's robot platform to handle natural, low-effort commands rather than requiring explicit verbal object names every time. If Samsung is building toward a household robot product — and its recent investments strongly suggest it is — gaze-based intent recognition is the kind of foundational capability that makes the difference between a robot that feels like a smart appliance and one that feels like a capable assistant.

Editorial commentary on a publicly published patent application. Not legal advice.