Sony Patent Triggers AR Visual Effects When Users Hit Their Mark
Sony has patented a system that watches for a colored light prop in a camera feed and fires a visual effect on screen the moment that prop reaches the right spot, no button press required.
How Sony's light-triggered AR overlay actually works
You're performing on camera and you want a burst of flames or a glowing aura to appear on screen the instant you raise a glowing wand to exactly the right position. Right now, pulling that off live usually means a crew member hitting a button at just the right moment, or expensive post-production work.
Sony's patent describes a system that handles this automatically. A camera feed watches for an object emitting a specific color of light (think a colored LED prop). Software tracks where that object is in the frame, and when it crosses into a target zone, an on-screen visual effect appears, superimposed over the live image.
No manual trigger needed. The prop's position is the trigger. That means performers can cue their own effects just by moving naturally, which could make interactive live video, themed entertainment, or streamed performances feel a lot more spontaneous.
a detection unit that detects a light emitter that emits light of a predetermined color from an image including a user carrying the light emitter …
Translation: It tracks a colored light carried by the user in the camera view.
How the system detects position and fires the effect
The system has two main jobs running in parallel.
First, a detection unit scans each frame of video for an object emitting light of a specific, pre-set color. Because the object is actively glowing (not just reflecting ambient light), it stands out reliably even in busy or changing backgrounds. The system tracks the pixel position of that light emitter across frames.
Second, a display control unit compares that detected position against a target zone defined in the image. When the relationship between the emitter's location and the target zone meets a set condition (for example, the prop enters a circular hotspot near the top of the frame), the system overlays effect information onto the video feed. That effect information is essentially a visual layer: fire, sparkles, a color wash, or any other pre-loaded graphic.
Key components described in the patent:
- A detection unit that isolates the colored light emitter from the rest of the image
- A display control unit that monitors positional relationships and decides when to fire
- A display unit that composites the effect layer onto the live camera feed in real time
The trigger logic is purely positional: the where of the prop determines the when of the effect, with no separate input device or crew member needed.
… causes the display unit to display effect information that is information indicating a predetermined visual effect in a superimposed manner on the image …
Translation: It triggers augmented reality visual effects on the screen when specific conditions are met.
What this means for live video and interactive content
For live performance, themed entertainment, or interactive streaming, the big cost has always been synchronization. A performer can't reliably hit a button while acting, and relying on a human operator introduces lag and error. Tying the visual effect directly to where a physical prop sits in the frame removes that coordination problem entirely. The performance itself becomes the control surface.
Sony operates across cameras, televisions, gaming hardware, and live production gear, so this kind of positional-trigger technology could surface in a wide range of products. AR stage kits, streamer accessories, or theme-park-style interactive cameras are all plausible homes for it. The new Big Tech patents Sony has been filing in the interactive display space suggest the company is methodically building the building blocks for performer-driven AR, piece by piece.
The design trades generality for reliability: requiring the performer to carry an actively glowing colored prop sidesteps the hard computer-vision problem of tracking arbitrary objects, but it also means the system only works with specific, prepared props in controlled lighting. That cost is real: the setup is never casual, and anyone who forgets to charge the LED wand breaks the whole effect. For a professional stage or a polished streaming setup where props are already part of the act, the constraint is minor and the reliability payoff is significant. For spontaneous home use, the prop requirement probably kills it before it starts.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0233089 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →