Google Patent Freezes Image Encoder While Training Text Model From Scratch

By Patentlyze Team · Updated Jul 21, 2026

Training an AI to understand both images and text at the same time is expensive and finicky. Google's new patent describes a shortcut: freeze the image side, train only the text side, and let one pull the other into alignment.

How Google's locked-image training method works

Imagine trying to teach someone to describe photos they've never seen. One approach: hire two teachers at once and have them learn together from scratch. Google's patent proposes something cheaper. Take a teacher who already knows images really well and lock them in place. Then hire a second teacher, focused only on text, and train them to match what the image expert already knows.

In practice, this means Google starts with an existing AI model that already understands pictures, then trains a brand-new text model to produce descriptions that line up with what the image model 'sees.' The image model never changes during this process. Only the text model learns.

The benefit is efficiency. You don't risk messing up a model that already works well, and you spend your computing budget only on the part that actually needs training. It's a bit like calibrating a new instrument against a known-good reference rather than rebuilding both from scratch.

How the fixed image encoder guides the text encoder

The patent covers a training technique for multimodal AI (AI that handles more than one type of input, such as images and text together). The core idea is called locked-model contrastive tuning.

Here's the setup:

• A pretrained image encoder (a model already trained to convert images into compact numerical summaries called latent representations) is loaded and its parameters are frozen. Frozen means the model's internal weights cannot change during the upcoming training run.
• A text encoder starts from scratch (untrained). Its job is to convert text strings into latent representations in the same numerical space.
• For each training example, both models produce their representations. A loss value (a score measuring how badly the two representations disagree) is calculated. Only the text encoder's parameters are adjusted to reduce that score.

The broader claim is written to cover any two data types, not just images and text. The 'first ML model' processes one data type; the 'second ML model' learns to match it on a different data type. That abstraction makes the technique applicable to audio-text, video-text, or other pairs as well.

What this means for cheaper, faster AI training

Training large multimodal AI models is one of the most expensive things a tech company can do. By locking the already-capable image model in place, Google sidesteps the risk of catastrophic forgetting (where a model loses old skills while learning new ones) and cuts the number of parameters that need updating, which reduces the computing cost of each training run.

For Google specifically, this kind of technique feeds into products like Google Lens, image search, and multimodal features inside Gemini. A faster, cheaper path to a well-aligned text-and-image AI means the company can iterate more quickly on those products without proportionally larger cloud computing bills, and that efficiency gap compounds over time.

Editorial commentary on a publicly published patent application. Not legal advice.