AMD Files Patent for Automatic Video Sorting That Trains AI to Animate Real-World Objects
Getting an AI to convincingly show a door opening or a flower blooming requires thousands of carefully labeled video examples. AMD's Xilinx division has filed a patent for a system that automatically hunts through raw video, cuts out the useful moments, and labels them so an AI can learn from them.
How AMD's system picks the right video clips for AI training
You're watching a cooking tutorial and every few seconds something changes: a pan heats up, butter melts, a dish comes out golden. Those individual moments of change are exactly what an AI needs to learn how objects behave over time, but combing through hours of video to find them by hand is exhausting work.
AMD's patent describes a system that does that combing automatically. It slices a video into short clips wherever something in the scene visibly changes state, scores each clip on sharpness, brightness, and how much motion actually occurred, and then discards the blurry, too-dark, or barely-moving ones. The clips that pass the test get short written descriptions automatically attached.
That curated collection then gets fed to an image-to-video AI model, training it to take a still photo and a text instruction like "let the candle melt" and produce a realistic short video of that happening. The idea is that better training data produces a more believable result for you as the end viewer.
partitioning a video into video segments corresponding to object state transitions depicted in the video; generating action instructions for each video fragment in a selected subset of the video segments based on the object state transitions depicted in the video segments in the subset; …
Translation: The system chops up footage wherever objects change and writes instructions describing those shifts.
How the pipeline scores and filters clips before model training
The patent describes a four-stage pipeline for building high-quality training data for a class of AI called image-to-video (I2V) generation models, which take a single still image plus a written instruction and output a short video clip.
- Segmentation: A video is automatically split into short clips at points where an object's visible state changes, for example a cup going from empty to full, or a light switching on.
- Quality scoring: Each clip is scored on several visual measures: similarity between the first and last frame (too similar means nothing interesting happened), average optical flow (a measure of how much pixel movement exists across frames), brightness, sharpness, and noise levels. Low-quality clips are dropped.
- Caption generation: The surviving clips are paired with auto-generated text descriptions called action instructions, derived from both the visual content and any existing text description of the original video.
- Model training: The curated clip-and-caption pairs are used to train the I2V model to perform what the patent calls Instructed Visual State Transition (IVST), following a text command to animate a believable physical change starting from a still image.
The quality filters are doing the real work here. By ruling out clips that are too dark, too shaky, or that barely change at all, the system keeps the training set focused on clear, unambiguous examples of cause-and-effect change.
The subset can be selected based on a measure of similarity between initial and final frames, average optical flows within the video segments, light levels, brightnesses, blurriness, or noise levels.
Translation: Clips are chosen for training based on factors like lighting, motion, and visual clarity.
What cleaner training data means for AI video generation
For anyone using an AI video tool, the quality of the output depends almost entirely on what examples the model learned from. If those examples were blurry, mislabeled, or ambiguous, the AI learns bad habits, and you end up with videos where objects morph strangely or motion looks unnatural. A pipeline that automatically filters for clean, well-labeled examples is a direct path to more believable output.
AMD keeps filing on AI video and generative model infrastructure, which signals that the company sees this as a meaningful area alongside its chip business. This particular patent sits one layer below the finished product: it is about the data factory that makes a good model possible, not the model itself. That is unglamorous work, but it is where a lot of real-world AI quality is actually decided.
That makes this the second AI vision filing from AMD we've tracked since September, following one on trimming unused AI code.
Every AI video tool that lets you say "turn this photo into a candle burning" lives or dies on the quality of its training examples. AMD's patent automates the process of finding and labeling those examples, which has historically been slow, expensive, and done by hand.
The person who benefits never sees any of this. What they see is a generated clip where the candle actually looks like it melts, rather than a blob of pixels that morphs strangely through the middle frames. That failure, the weird in-between moment, is exactly what bad training data produces.
If the system works, cleaner data means fewer of those artifacts, and the improvement shows up in tools people already use.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0279048 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in