Google Files Patent for an AI That Erases Text From Photos for Translation
Translating a sign in a photo means first erasing the original words, and Google's filing teaches an AI to do that by writing its own answer key. The trick avoids the need for a pile of hand-made before-and-after photo pairs.
What Google's photo text-eraser actually does
Every time you aim your phone at a foreign menu or street sign, a translation app has to do two jobs: read the words, then paint your language over the top. The second job is the messy one, because the old words have to vanish without leaving a smudge.
Teaching an AI to erase text cleanly normally takes thousands of photos paired with a text-free twin, and those pairs are hard to find. Google's filing gets around that by making its own. One image AI first practices filling in blank patches that contain no text, so it learns to recreate backgrounds instead of inventing new letters. Then it cleans the text out of real photos, producing the missing twins.
A second AI trains on those pairs, and a small version of it is built to run on a phone. The filing mentions a target of under 50 milliseconds per image.
… processing, by the computing system, a plurality of first images that comprise text with a fine-tuned version of the first pre-trained diffusion model to generate a plurality of augmented first images, wherein the plurality of augmented first images are descriptive of the plurality of first images with the text replaced with predicted background pixels …
Translation: The AI removes words from photos and fills in the blank spaces with what should naturally be in the background.
How Google trains one AI to teach another
The filing covers a multi-stage way to train a text-removal model. It starts from general-purpose diffusion models (image generators that build or repair a picture by gradually cleaning up noise) that were already pre-trained on huge image collections.
The core claim has four steps:
- Blank out patches of training images that contain no text (found with OCR, or optical character recognition, software that spots lettering), then fine-tune a first model to rebuild those patches.
- Run that model over real photos with their text masked, producing copies where the text is replaced with predicted background pixels.
- Fine-tune a second model using the original photos as input and the cleaned copies as the answer key. It sees the whole photo, with no blanked-out areas.
- Optionally, shrink it through knowledge distillation (a small "student" model learns to copy a big "teacher" model's outputs).
The compact model takes a photo's three color channels plus a fourth channel holding a black-and-white mask that marks which text to remove. The mask works as a hint, and the pixels under it stay visible, so the model still sees the whole scene. That lets it remove one word from a sentence and leave the rest.
The filing says about 2,000 training samples may be enough for the first model, and it describes a target of 50 milliseconds or less per image.
… fine-tuning a second pre-trained diffusion model using the plurality of first images and the plurality of augmented first images, the second pre-trained diffusion model configured to reconstruct the plurality of synthetic training data items and the second plurality of training data items without mask inputs …
Translation: A second AI model learns to clean up the images completely without needing any human guidance on where the text is located.
Why live camera translation gets a cleaner look
If you use camera translation, this is about how natural the result looks. Cleaner erasing means translated signs, menus and book covers could keep their shadows and textures instead of showing a smeared box. The filing names live camera translators on phones, wearables and AR glasses as targets, where even a fraction of a second of delay would be noticeable.
For Google, the interesting part is the training shortcut. Real before-and-after photo pairs barely exist, so the system writes its own answer key, with an option for human reviewers to pick the best candidates. The filing also says the same approach could apply to removing objects or restoring images, though it lists that only as a possibility.
Google's 81st filing we've tracked since May in the AI photo editing race adds to a run that includes one fixing blurry photos and one catching document glare.
If you have ever aimed a translation app at a menu and seen the new words sit on a smeared blotch, you know what Google is trying to fix. When this works, a sign in another country looks as if it was printed in your language, with the shadows and textures intact. When it fails, you see ghost letters or a blurry patch where the sign used to be.
Nothing here gives you a new button. It is plumbing that decides whether an existing feature looks polished or sloppy, and you would notice it only by never noticing the erasing at all.
The filing gives under 50 milliseconds per image as a target, which is quick enough that you would not feel a delay. Whether real phones hit that is something only a shipped product could show.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0350763 A1 · click any drawing to enlarge
Show the other 2 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in