Sony Patents a System That Color-Codes Each Performer's Part in Live Music Videos
Imagine watching a live band performance and instantly knowing which singer is carrying the melody right now — because their lyrics glow in their own color on screen. That's the core idea behind Sony's latest patent.
What Sony's color-coded music video overlay actually does
Picture watching a K-pop group perform live. Four members are singing at once, harmonizing and trading lines, and unless you already know the song well, it's hard to track who's singing what. Sony's new patent describes a system designed to fix exactly that.
The idea is straightforward: as a performance video plays, the system listens to the audio coming from each performer and highlights the matching lyrics — or a symbol representing that person — in a color assigned to them. When member A sings, their text lights up in blue; when member B takes over, their text switches to red.
It works across multiple performers and multiple video feeds at once. Sony describes it as applying to any setup where several "sound sources" (singers, instruments, speakers) are captured together, making it easier for you to follow along with who is doing what in real time.
How the system matches sound sources to colored text overlays
The patent describes an information processing device that takes two inputs: text data representing the sounds coming from multiple sources (think synced lyrics or musical notation), and one or more video feeds that include those sources as visual objects.
From those inputs, a video generation unit builds what Sony calls a "presentation video" — essentially the original footage with an overlay added. That overlay uses color as the primary signal. Each distinct sound source gets its own color, and as the audio plays:
- The portion of text (lyrics, notation) matching what's currently being performed lights up in that source's color
- A figure or character sequence — like an icon or name tag — representing that performer can also appear in the same color
- The color updates in real time as performers trade off lines or harmonize
The system is built to handle multiple simultaneous sound sources, meaning it can in theory track a full ensemble rather than just a single lead. The patent doesn't specify exactly how the audio separation is performed, but the claim assumes the system can correctly identify which source is producing sound at any given moment and map that back to the right visual marker.
What this means for concert broadcasts and karaoke-style apps
For fans watching concert broadcasts, music documentaries, or streaming performances, this kind of overlay could make dense ensemble performances far easier to follow — especially for groups with many members singing different parts. Think of it as a real-time "who's singing now" guide built directly into the video.
For Sony specifically, the technology sits neatly alongside its existing businesses in music (Sony Music), professional audio and video equipment, and entertainment content. A system like this could show up in broadcast production tools, streaming apps, or even consumer karaoke software — anywhere that syncing performers to their parts in a visually clear way adds value for your viewing experience.
This is a modest but genuinely useful idea — the kind of thing that fans of large ensemble acts like orchestras, choral groups, or idol groups would actually appreciate. It's not a technical moonshot, but it solves a real problem in an intuitive way. Whether Sony builds it into a product or it quietly sits in a patent portfolio is the real question.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.