Microsoft Files Patent for Sharpening Low-Quality Video Without Lag
AI video sharpening looks great but is too slow to run on every frame of a 60-frames-per-second video. Microsoft's patent runs the heavy AI step only now and then and reuses its results in between.
What Microsoft's video-sharpening patent does
Imagine you're watching a 360p video on a laptop and you want it to look like 1080p. AI can sharpen video that way, but it is so slow that running it on every single frame can't keep up with smooth 60-frames-per-second playback.
Microsoft's patent describes a shortcut. The slow AI sharpening runs on an occasional frame, and that sharpened picture is then reused for the frames that follow, wherever nothing important has changed. A motion check works out where things moved, so the old sharp pixels can be slid into the right spot.
Where the shortcut would look wrong, such as a new object entering the scene, the system falls back to the plain pixels from the current frame. You get a mostly sharp picture at a speed the AI alone couldn't manage.
… generating a pixel source map identifying: first regions of an output frame to be generated using image data from the reference super resolution frame, and second regions of the output frame to be generated using image data from the current low resolution frame; …
Translation: The system creates a map that decides which parts of the video are enhanced and which parts stay normal.
How the pixel source map picks each pixel
The system keeps three things on hand: the current low-resolution frame, an earlier low-resolution frame, and the AI-sharpened version of that earlier frame (the "super resolution" frame). It first computes motion vectors, which are small arrows describing how each patch of the picture shifted between the two low-resolution frames. The filing says these can come from the video hardware in a graphics chip, with no help from the video file format itself.
Next it decides, region by region, whether the old sharpened frame can be trusted. It compares blocks of pixels (8x8 or 4x4) using a sum of absolute differences (basically adding up how different the pixels are after the motion adjustment). A block that matches well is accepted as is. For a poor match, individual pixels are checked one by one.
The result is a pixel source map: a grid of values from 0 to 1 saying whether each pixel should come from the old sharpened frame or from the current frame. A Gaussian filter (a standard blur) softens the edges between the two kinds of regions so no seams show.
The filing also describes two supporting pieces:
- A color lookup table, learned from the AI's output, that nudges plain pixels toward the AI's color adjustments so the patches blend in.
- A scheduler that runs the full AI model in the background, only when no earlier sharpening request is still pending.
If the reference pixel equals the corresponding input pixel, the reference super resolution result pixel is rendered for this pixel, and otherwise a corresponding pixel from the current input frame is rendered.
Translation: If pixels match between frames, the system uses the high quality version, but otherwise it uses the standard video frame.
Why this matters for streaming on weaker devices
If you watch lower-quality video on a laptop, phone or other everyday device, the practical question is whether sharpening can run live without stuttering. The filing says a 60-frames-per-second video leaves about 16 milliseconds per frame, and that shrinking the AI model to fit makes results drift toward plain old upscaling. This patent goes after the problem differently: do less AI work, not a weaker version of it.
The approach also works with any video format, because it recomputes motion from the decoded picture instead of changing each codec. For Microsoft, that fits Windows-style hardware, where the filing points to graphics-chip functions already exposed through DirectX 12. It is a patent application, so none of this says a feature is coming.
Microsoft's eighth filing we've tracked since July in our AI photo editing race builds on earlier applications to rewrite recorded video speech and rewrite live video lips.
The problem here is everyday and expensive in a way people feel. A huge amount of video is watched at lower quality than screens can show, and sharpening it with AI only helps if it keeps pace with playback. Miss the 16 millisecond budget for a frame and you get stutter, which is worse than a soft picture.
The approach fits that problem well. Instead of shrinking the AI until it stops being useful, the filing reuses finished work wherever the picture hasn't changed, and it admits where reuse fails: rotating cameras, zooms and new objects such as a traffic light switching color.
The fallback to plain pixels for those moments keeps the shortcut honest. The patch-hiding color table and blur show the authors thought about what viewers would actually notice. It is a practical fix to a real bottleneck, and the size of the fix matches the size of the problem.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0350748 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in