Google Patents a Neural Network That Colorizes Old Photos of People
Old black-and-white photos are notoriously hard to colorize convincingly — skin tones, hair, and clothing all behave differently. Google's new patent describes a CNN that understands which body part it's looking at before it picks a color.
What Google's person-aware colorization CNN actually does
Imagine scanning an old family photo from the 1940s — grainy, black-and-white, a grandmother you never met. Automatic colorization tools often get it wrong: faces end up with greenish skin, or a dark jacket looks the same as dark hair. Google's patent describes a smarter approach that first figures out what part of a person it's looking at — face, hair, clothing — before deciding what color each pixel should be.
The system is trained on color photos paired with their grayscale versions, plus labels that identify body parts. That extra context teaches the model to treat a cheek differently than a collar.
A built-in "discriminator" acts like a skeptical judge: it looks at every pixel and asks, "does this look like a real color photo, or an AI guess?" The network keeps adjusting until the judge can no longer tell the difference.
How the pixel discriminator trains Google's colorization model
The patent describes a convolutional neural network (CNN) — a type of image-processing AI that learns by seeing millions of examples — trained specifically to colorize grayscale images that contain people.
The training pipeline has a few notable pieces:
- Part segmentation: Before colorizing, the CNN identifies body-part regions (skin, hair, clothing, etc.), so it can apply contextually appropriate colors rather than guessing blindly.
- Paired training data: The model is trained on real color photos alongside their grayscale equivalents, plus part annotations — essentially maps that label which pixels belong to which body region.
- Pixel discriminator: A secondary network that evaluates the colorized output pixel by pixel and predicts whether each one came from a real photo or the model's output. This is a form of perceptual loss — a training signal that pushes the model toward photorealistic results rather than just technically close color values.
The output of the CNN is a feature vector (a numerical description of the colorized image) that simultaneously encodes both color predictions and part-segment predictions. Modifying the CNN's parameters based on the discriminator's feedback is a classic GAN-style training loop (Generative Adversarial Network), where generator and critic improve each other iteratively.
What this means for Google Photos and legacy image restoration
For Google Photos, which already offers editing and memory features, a person-aware colorization model could meaningfully improve the quality of automatic restorations for old family photos — one of the most emotionally resonant use cases in consumer AI. The part-segmentation approach is the key differentiator: it means skin tones and fabric colors are treated as separate problems, which is how a human retoucher would actually think about it.
Beyond nostalgia, accurate colorization of people has downstream value in media archiving, historical digitization projects, and creative tools. The discriminator-based training is not novel on its own, but applying it with body-part awareness to person-centric images is a focused, practical improvement on general colorization methods.
This is a focused, well-scoped engineering patent — not a moonshot. The part-segmentation twist is genuinely clever and addresses a real failure mode in existing colorization tools. If Google ships this into Photos or a Pixel feature, most users won't know what changed, but they'll notice their grandmother looks less like she has a skin condition.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.