Google Patents Training Image AI on Photos of Every Shape and Size at Once
Most image AI learns from photos squeezed into identical squares. Google's patent application lets it learn from several photos at once, whatever their shape.
What Google's mixed-size photo training does
Most image AI is trained on photos that have all been resized into the same square shape first. A tall phone snapshot and a wide landscape both get squeezed to fit, so the AI never sees them as they really are.
Google's patent application describes a different way to feed the training. Each photo is cut into small tiles, and tiles from several photos are lined up in one long row that the AI reads in a single pass. It is a bit like fitting several small parcels into one box instead of shipping each in its own.
The AI is kept from mixing up the photos, so each one is judged on its own. The filing says this lets a single model handle pictures of different sizes later, and that it can match the quality of older methods with about four times less computing.
dividing the training image into a respective plurality of patches; and generating a respective token representing each of the patches; generating one or more input sequences, wherein each input sequence corresponds to two or more of the training images in the batch …
Translation: The system chops up multiple photos into smaller pieces and combines them into single streams of data.
How Google packs several photos into one row
The patent covers a way to train vision models that cut each image into patches (small square tiles) and turn each patch into a token (a numeric stand-in the model can read). Popular designs such as the Vision Transformer (ViT, an image-recognition model that reads tiles the way a language model reads words) normally need every image resized to a fixed shape and a fixed number of tiles.
Here, tokens from two or more images are packed into one input sequence. A large image produces more tokens than a small one, so images with different resolutions and aspect ratios can share a sequence. The filing describes a greedy fill: go through the images, place each in the first sequence with room, and start a new sequence if none fits. Blank filler tokens pad each sequence to the same length, which suits chips such as TPUs that prefer fixed-size inputs.
The filing describes three supporting pieces:
- Masked attention: self-attention (the step where tiles look at each other) is blocked across images, so tiles only learn from their own photo.
- Token dropping: some tiles are randomly removed to speed up training, and the rate can differ per image or change over the course of training.
- Resolution sampling: each image is rescaled to a sampled size while keeping its aspect ratio.
Training scores the model with a loss function (a measure of how wrong it is), for example by matching images to captions or by classifying objects. The trained model can then be adapted to tasks like object detection or image captioning.
… processing each input sequence using the image processing neural network to generate a respective training output for each corresponding training image; and training the image processing neural network on a first loss function …
Translation: The neural network processes these mixed sequences all at once to learn how to analyze images properly.
What cheaper image training means for Google AI
Training big image models eats a huge amount of computer time, and the filing puts a number on the savings. It says the approach can match a standard ViT trained with earlier techniques using 4x less compute. Combined with mixed sizes and token dropping, it says the system can process five times more images during training.
There is also a benefit for the finished model. Because it has seen many sizes and shapes in training, one model can handle different resolutions afterward, so your photos do not all need to be squashed to a standard size first. Anything that recognizes or describes pictures could use a model trained this way, though the patent names no specific product.
Google's 45th entry in the AI training & infrastructure filings we've tracked since May builds on earlier work like one predicting workload performance and one on light changing photos.
The problem is real. Training an image AI takes enormous amounts of computer time and electricity, and forcing every photo into a square has always thrown away information, since a panorama and a portrait are very different shapes. A method that reaches the same result with a quarter of the computing goes after a genuine cost.
The fix fits the size of the problem. It reorganizes how photos are loaded and grouped rather than redesigning the AI itself, which keeps the idea simple and makes it usable on any model trained this way.
You will never see it on your phone. The four-times figure comes from Google's own filing, so treat it as the company's claim until outside teams test it. The idea is sensible, and the cost it targets is large.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0311467 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in