Google Files Patent for a Tweak to the Math Inside AI Language Models
Google's patent covers one small step that repeats all through an AI language model: instead of one filter and a switch, it runs two filters side by side and multiplies their results.
What Google's two-path AI layer actually does
Every time you translate a sentence, dictate a text or ask a chatbot a question, the AI behind it passes your words through the same short routine, again and again, layer after layer. One step in that routine works like a mixing station, where each word's information gets reshaped before moving on.
Google's patent swaps that mixing station for a different design. The usual version runs each word through one filter and then a switch that cuts off weak signals. Google's version runs the same word through two separate filters at once, puts a switch on one of them, then multiplies the two results together. In effect, one filter proposes something and the other decides how much of it gets through.
Google says the result is a model that handles text, speech and other sequences more accurately, including when it was first trained on piles of unlabeled data.
… applies an element-wise multiplication between two vectors generated as a result of two different linear transformations performed on the same attended layer input.
Translation: It multiplies two altered versions of the data together inside the model's neural layers.
How the two filters get multiplied together
The patent covers an attention neural network (the design behind most modern chatbots and translators). It stacks many layers, and each layer has two parts. An attention sub-layer lets every word in a sequence look at the other words to pick up context. A feed-forward sub-layer then reshapes each word's data on its own.
The claim is about that second part. For each word position, the feed-forward sub-layer does the following:
- Applies a first linear transformation (multiplying the data by a table of learned numbers), then a non-linear activation function (a switch that bends the result so the network can learn more than straight-line patterns).
- Applies a second, different linear transformation to the same input.
- Multiplies the two results together element by element (first number with first number, second with second, and so on).
- Uses that product to produce the layer's output.
Both transformations are learned during training, and the same two are reused at every position in the sequence. The claim also says the network is trained on an unsupervised data set (large piles of data with no human labels).
The description adds a third transformation after the multiplication, plus a residual connection (a shortcut that carries earlier data forward) and layer normalization (keeping the numbers in a consistent range). It lists ReLU, sigmoid, GELU and Swish as possible switches.
… generating a third transformed input by performing an element-wise multiplication between the first transformed input and the second transformed input; …
Translation: The system multiplies two processed vectors together to change how the neural network handles data.
Why a small AI tweak can reach many tasks
If it works as described, you would never see this change directly. It would sit inside the AI models that translate your messages, turn your voice into text, spot a wake word, answer your questions or describe an image. The filing also lists health predictions, robot control and DNA analysis as tasks the same layer could serve.
The filing gives no test scores and names no products, so how big the gain is stays unclear. What it does show is a basic building block that any attention-based model could use. Because the change is pure software, it could go anywhere an attention network already runs.
That makes this Google's 113th filing we've tracked in Language AI since May, a body of work that includes applications on keeping search prices current and catching near matches in AI search.
This sits about as close to shippable as a patent gets, because it needs no new hardware. It is a change to the math inside one layer of a language model, so the shortest route to a product is a code edit and a fresh training run.
The training run is the cost. The two filters are learned, so a model already built the old way would likely have to be trained again from the start, which takes time and a lot of computing power at the scale of a big model.
The filing offers no accuracy numbers, so the size of the payoff is the open question. The idea itself is simple enough to explain in one breath, which makes it easy to adopt: swap one layer, retrain, and measure.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Google patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
3 drawing sheets from US 2026/0340390 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in