Get the newsletter
Qualcomm · AR / VR · Filed Mar 26, 2025 · USPTO published Oct 1, 2026 · Oct 1 release

Qualcomm Patents a Way for AR Glasses to Answer Questions About Any Object You Point At

What if you could tap on any unfamiliar object in an AR headset and just ask it a question? Qualcomm has filed a patent for exactly that, using a language model to generate context about real-world objects the device has never seen before.

A person wearing AR glasses asks questions about a robot arm, receiving answers based on generic or specific knowledge. Drawing from patent filing US 2026/0301329 A1.
A person wearing AR glasses asks questions about a robot arm, receiving answers based on generic or specific knowledge.All 11 drawings ↓
Publication number
US 2026/0301329 A1(US20260301329A1)
Applicant
QUALCOMM Incorporated
Filed · Published
Mar 26, 2025 · Oct 1, 2026
Inventors
Robert Peter VIEHAUSER, Markus EDER, Markus Stefan BIRKNER, Horia-Mihai BUJANCA
Status when we published
Examiner raised objections; the company can respond
Document
20 claims
US classification
345/633
Examiner
OCHSNER, ISABELLA PAIGE (Art Unit 2618)

What Qualcomm's point-and-ask AR system actually does

Imagine you're wearing AR glasses in a factory, and you spot a piece of machinery you've never seen before. You tap on it and ask, "What does this do?" The glasses don't have a pre-loaded database of every machine on earth, but Qualcomm's patented system figures it out anyway.

Here's the trick: when you tap on a spot on that object, the glasses know exactly where your finger pointed in the room. The system then converts that location into the object's own internal coordinate system, so it always knows which part of the object you touched, even if the object moves or rotates. That precise location, paired with your question, gets handed off to an AI language model.

The language model then generates a relevant answer based on the domain you're working in, whether that's manufacturing, medicine, or retail. It's less about recognizing the object from a library and more about letting AI reason through an unfamiliar situation using the spatial context you gave it.

From the filing · CLAIM 1
… transform, based on a pose between the world coordinate system and the local object coordinate system, the first set of coordinates that represent the second user input in the world coordinate system to a third set of coordinates in the local object coordinate system; …

Translation: The system maps where you pointed in the real world to the exact spot on the 3D object model.

How the coordinate translation ties your tap to the object

The patent describes a processing system, likely inside an XR headset or a chip powering one, that handles three kinds of input at once:

  • A verbal or text query from the user, tied to a specific domain (like "maintenance" or "retail")
  • A spatial tap or pointer that identifies a location on a real-world object
  • Object state information, which includes the object's own internal coordinate grid (called a local object coordinate system) and its orientation in space

The core engineering challenge here is a coordinate translation step. When you tap a spot in the real world, that location is recorded in world coordinates, meaning a fixed grid tied to your environment. But objects move. If someone rotates a box or a robot arm shifts, world coordinates alone won't tell you which part of the object you touched.

The system solves this by computing the pose (the position and rotation relationship) between the world coordinate system and the object's own local coordinate system. It transforms your tap's world coordinates into the object's local coordinates. That locked-in local position is then paired with your query and passed to a language model (an AI similar in concept to what powers chatbots), which generates a contextually relevant response.

The domain parameter matters here. The same tap on the same object could produce a very different answer depending on whether you're asking as a technician, a shopper, or a doctor. The system routes the language model's reasoning through that domain lens.

From the filing · THE ABSTRACT
… generating, with a language model, an output based on the first user input, the first association, and context associated with the domain.

Translation: An AI language model uses your question and the specific pointed spot to generate an answer.

What this means for AR headsets that meet the real world

AR headsets have always struggled with the gap between the digital and physical worlds. Most current systems work by pre-labeling known objects, which means anything outside the database gets ignored. Qualcomm's approach flips that assumption: instead of requiring prior knowledge, it uses spatial precision plus AI reasoning to handle anything a user points at.

For industries like manufacturing, logistics, or field service, this could mean workers get instant, context-aware guidance on unfamiliar equipment without a pre-built knowledge base for every machine. For consumers, it inches AR glasses closer to a genuinely useful tool rather than a novelty. the pattern in Qualcomm's XR and on-device AI filings suggests the company is betting that the hardware intelligence layer, not just the display, is where AR wins or loses.

Qualcomm's 51st filing we've tracked since July in the AR glasses race adds to a pattern that includes focused hologram delivery and GPS-locked spatial audio.

Source: US 2026/0301329 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Qualcomm patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 11 sheets

11 drawing sheets from US 2026/0301329 A1 · click any drawing to enlarge

AR/VR devices and network., drawing sheet 1 from US 2026/0301329 A1
Sheet 1 AR/VR devices and network.
AR glasses answering object questions., drawing sheet 2 from US 2026/0301329 A1
Sheet 2 AR glasses answering object questions.
Context generation for AR interaction., drawing sheet 3 from US 2026/0301329 A1
Sheet 3 Context generation for AR interaction.
Coordinate systems for object tracking., drawing sheet 4 from US 2026/0301329 A1
Sheet 4 Coordinate systems for object tracking.
Output generation for AR interaction., drawing sheet 5 from US 2026/0301329 A1
Sheet 5 Output generation for AR interaction.
AR output display and augmentation., drawing sheet 6 from US 2026/0301329 A1
Sheet 6 AR output display and augmentation.
Machine learning model training., drawing sheet 7 from US 2026/0301329 A1
Sheet 7 Machine learning model training.
Neural network architecture., drawing sheet 8 from US 2026/0301329 A1
Sheet 8 Neural network architecture.
Show the other 3 sheets
Qualcomm Patents a Way for AR Glasses to Answer Questions About Any Object You Point At patent drawing, sheet 9
Sheet 9
Qualcomm Patents a Way for AR Glasses to Answer Questions About Any Object You Point At patent drawing, sheet 10
Sheet 10
Qualcomm Patents a Way for AR Glasses to Answer Questions About Any Object You Point At patent drawing, sheet 11
Sheet 11
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.