Intel Patents a Way to Blend AI-Generated Video Into Live Camera Footage in Real Time
Intel has filed a patent describing a way to weave AI-generated visuals directly into a live camera feed, frame by frame, on the chip itself, without shipping the video to the cloud and waiting.
What Intel's camera-plus-AI pipeline actually does
Every time your phone's camera shows you a picture on screen, a special chip inside the phone instantly cleans up and sharpens that picture before you ever see it. That chip works very fast. The AI tools that generate or alter images work much more slowly and have always run as a completely separate process.
Intel's patent describes a way to connect those two worlds. Two small AI programs sit inside that fast chip and act as translators. The first converts the camera's live picture into a format that a powerful image-generating AI can read and work with. The second converts the AI's output back into a normal picture. The AI can add objects, swap out backgrounds, or change the look of a scene. The altered picture then flows back into the camera's normal process as if nothing unusual happened.
To keep up with the speed of live video, the system does not run the heavy AI tool on every single frame. Instead it fills in the gaps between AI-processed frames and skips re-working parts of the scene that have not changed. The result is live, AI-altered video, processed entirely on the device itself.
… a latent projector configured to receive image data from the first processing block and project the image data from an image space representation to a latent space representation, wherein the latent space representation corresponds to the latent space of a diffusion model …
Translation: It translates regular camera video into the special mathematical format that AI image models use.
How the latent projectors slot into the ISP pipeline
The patent centers on inserting two lightweight neural networks, called a latent projector and a latent deprojector, into the middle of an image signal processor pipeline.
- Latent projector: Takes the ISP's partially processed image and maps it into "latent space" (the compressed, abstract numerical format that diffusion models work in internally). Think of it as translating a photograph into the private shorthand a particular AI uses to understand images.
- Latent merge block: Combines that translated image with whatever the diffusion model has generated or modified in that same shorthand, producing a blended representation that contains both the real camera data and the AI's contribution.
- Latent deprojector: Translates the merged shorthand back into a normal image and hands it to the rest of the ISP pipeline for final output.
The projector and deprojector are described as compact convolutional neural networks (a class of image-processing neural network well suited to running on hardware accelerators) trained to approximate the encoder and decoder of a variational autoencoder (the compression layer inside most diffusion models). Being compact is the whole point: the heavy diffusion model runs less frequently, and the lightweight projectors handle the per-frame translation work.
Three mechanisms help hit real-time frame rates: keeping most computation in compressed latent space rather than full-resolution image space; temporal subsampling (running the expensive AI model only every few frames and interpolating the gaps); and static background masking (skipping AI processing on parts of the frame that haven't changed).
Real-time operation is enabled through latent-domain computation, temporal subsampling of the diffusion model with latent-space interpolation, and static background masking …
Translation: The system runs fast enough for live video by skipping frames and ignoring unchanging backgrounds.
What this means for on-device AI cameras
AI image generation is computationally expensive in a way that has so far made it incompatible with live video on a camera chip. The workarounds have generally involved either a cloud round-trip (adding latency and bandwidth costs) or a separate, powerful GPU. Intel's approach, if it works as described, would let a device apply diffusion-model-style effects to a live feed using hardware close to what already exists in phones, laptops, and dedicated cameras.
For your devices, that could mean real-time AI background replacement, style transfer, or scene augmentation that runs entirely locally, without a network connection and without the delay that makes current AI video tools feel laggy. Intel's filing sits squarely in the on-device AI processing space that new Big Tech patents across the chip and camera industry have been converging on, as companies race to move inference workloads off the cloud and onto the sensor itself.
Intel's fifth filing we've tracked in our AI photo editing race since July builds on earlier applications like one on isolating objects in photos and one on smarter video upscaling.
The problem here is real: AI video tools are too slow and too power-hungry to run inside the chip that processes your phone camera's image. Intel's solution is to train smaller, faster stand-in networks that mimic the big AI model closely enough to get the job done frame by frame.
The trade-offs are honest engineering, not magic. The system skips processing some frames and ignores the background, which saves time but raises real questions about picture quality. Intel's own patent admits the stand-ins do not perfectly copy the full model, and that gap is where the results will either hold up or fall apart.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0245167 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →