Microsoft Patents Video That Lets Viewers Change What They See While Watching

By Patentlyze Team · Updated Jul 21, 2026

What if you could swap out elements in a video while it was playing, and the file would remember your changes without breaking on any standard player? That's what Microsoft is describing in this patent.

What Microsoft's interactive video format actually does

Imagine watching a product demo video and being able to click to change the color of the item on screen, or watching a tutorial and switching between two different step-by-step versions without the video stopping or buffering. Microsoft's patent describes a system that would let exactly that happen.

The trick is that the video file carries two layers at once: the original, unmodified version of the video and the viewer-modified version. When you make a change during playback, your version gets saved into the file's metadata (hidden data that travels with the video). Both versions can play side by side inside the file.

Critically, the file is designed to work in existing video players, even ones that don't know anything about this interactive feature. A player that doesn't support the new tricks just plays the original version as normal. One that does support it can show you the interactive experience.

How the dynamic and static layers render in parallel

The patent describes a pipeline with two main phases: authoring (where a video editor sets up which elements can be changed and how) and playback (where a viewer actually makes those changes in real time).

During authoring, the video editor identifies dynamic variables (think: the color of an object, the text on a banner, the language of subtitles) and links each one to a specific interaction a viewer can perform, such as clicking, sliding, or selecting from a menu. Those interaction rules get written into the video file's metadata, which is the hidden information layer every video file already carries for things like resolution and frame rate.

During playback, when a viewer triggers one of those interactions, the video player does two things simultaneously:

• It generates a modified version of the affected content and embeds it into the metadata of a new version of the file.
• It keeps the original, static content intact so the two can render in parallel.

The key engineering bet here is backward compatibility: the file format is designed so that standard players (ones that have never heard of this system) simply ignore the dynamic layer and play the original. Only players built to support the feature can surface the interactive experience. This avoids the classic problem of new formats breaking on older software.

What this means for streaming and interactive content

For viewers, the immediate promise is richer online video: product configurators, training materials, or entertainment that responds to your choices without requiring a full app or a browser-based workaround. The fact that your modifications travel inside the video file itself means the experience could theoretically move with you across devices or be shared with someone else in a personalized state.

For Microsoft, this looks like a building block for platforms like Teams, SharePoint, or Clipchamp, where interactive video in enterprise or creative contexts would be a natural fit. The backward-compatibility angle is the pragmatic part: it means adoption doesn't require everyone to upgrade their software at once, which is usually what kills new media formats before they get traction.

Editorial commentary on a publicly published patent application. Not legal advice.