Apple Patents Floating Controls for Moving and Resizing Virtual Objects by Looking and Pinching
Choosing whether to move, spin or resize a virtual object could come down to a glance at the right option, then a pinch and a hand movement.
What Apple's look-and-pinch object controls do
Moving things around in a headset is harder than it sounds. Apple's own filing calls many current approaches cumbersome, tedious and error-prone, because sliding, spinning and resizing a virtual object can each call for a different input you have to remember.
The patent proposes a simple pattern. When you look at a virtual object and hold part of your body, such as a hand, in a certain pose, a small set of options appears, one for each way of handling the object. You look at the one you want, pinch to select it, then move your hand. The direction and distance of your hand's movement set the direction and amount of change in the object.
Once you have chosen, you can look away and the action carries on. The options can also follow your hand or the object around, and they disappear when you stop holding that pose.
… manipulating the virtual object according to the first input mode in accordance with one or more parameters of the second input; and in accordance with a determination that the first input was detected while the attention of the user was directed to the second representation of the second input mode, manipulating the virtual object according to the second input mode …
Translation: The system changes how it moves the object depending on which control you looked at first.
How a glance picks the way you move an object
The core of the filing is a screen layout. A virtual object appears alongside two or more small on-screen choices, each standing for a different way of manipulating it (the claims name moving, rotating and scaling, which means resizing). The system tracks your attention, which the filing mostly treats as where you are looking, though it can also be where a handheld device is pointing.
Choosing takes two steps:
- Select: a selection input, such as a pinch or a button press on a handheld device, happens while your attention is on one of the choices. That locks in the mode.
- Act: a second input, such as a hand movement, changes the object in that mode. Bigger movements make bigger changes, and the direction of the movement sets the direction of the change.
The filing adds some forgiving details. Once you have chosen, the change continues even if your gaze drifts away. If your gaze leaves the choice before your selection finishes, the system still counts it when the selection ends within a short time limit, and ignores it if it ends later.
The options appear only when you look at the object while a part of your body, such as a hand, is in a particular pose. The description also covers direct input (reaching out to the object itself) and indirect input (looking at it while gesturing from wherever your hand happens to be).
In some embodiments, a computer system manipulates a virtual object based on input from a hand of a user and/or a handheld device.
Translation: The device tracks your hand movements or a controller to let you interact with virtual items.
Why this could make headset editing less fiddly
If you wear a headset, the practical payoff is fewer steps and fewer mistakes when you handle 3D things like a product model, a room layout or a piece of art. A look picks the action, a pinch confirms it, and your hand supplies the movement. Because the choices appear only while you need them, your view stays uncluttered.
The filing describes a head-mounted device with eye-tracking and hand-tracking cameras, and it says cutting the number of inputs saves effort and battery. Interaction rules like these shape whether spatial computing feels natural, and small fixes in them add up to the difference between a device you reach for daily and one that feels like work.
Apple's 40th filing we've tracked since May on our eye and hand controls watchlist follows a food tracker patent and arm-sensitive glasses.
This one sits close to shippable. Nothing in the filing needs new hardware: it relies on eye tracking, hand tracking and a display, all of which the document describes as ordinary parts of a headset. The work is software, a menu that appears when you look at an object and raise a hand, plus rules for what a pinch and a drag then mean.
The shortest route to a product is a system update for a headset that already follows your eyes and hands. What has to come first is tuning. The claims spend real effort on edge cases, like what happens if your eyes drift off the menu mid-choice, and that detail decides whether this feels natural or fiddly. The paperwork traces back to 2022 filings, so the idea has had years to mature.
Expect a small quality-of-life upgrade. If it works, you will mostly notice that moving a 3D object stops feeling like work.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Apple patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
59 drawing sheets from US 2026/0310304 A1 · click any drawing to enlarge
Show the other 51 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in