OpenAI Patents a Chip That Runs AI Calculations Forward and Backward Without Interruption

By Patentlyze Team · Updated Jul 31, 2026

Moving data around inside a chip is often more expensive, in time and energy, than doing the actual math. OpenAI's new patent describes hardware that sidesteps a common version of that problem by running two different flavors of matrix multiplication on the same weights without ever shuffling them.

See all 6 drawings from this filing ↓

What OpenAI's stationary-weight chip trick actually does

Imagine a massive spreadsheet of numbers permanently bolted to a wall. Normally, to do two different calculations with it, you'd have to print two separate copies, one in the original layout and one with all the rows and columns swapped. That copying takes time and wastes energy.

OpenAI's patent describes a chip design that keeps those numbers exactly where they are and instead changes how the chip reads and combines them. A routing switch inside the hardware directs partial results to different adder circuits depending on which kind of calculation you need, original layout or flipped.

Almost every AI model, including the large language models behind ChatGPT, runs on enormous amounts of this kind of matrix math. Doing it faster and with less data-shuffling means lower costs and potentially lower power draw for the servers that run these models.

How the multiplexer routes results for two matrix operations

The patent covers a hardware unit that can perform two related operations, a standard vector-matrix multiplication (VMM) and a transposed VMM (where the matrix is flipped so its rows become columns), without ever moving the stored weight values.

The key components described are:

• Memory cells that hold the weight values and stay put throughout both operations
• Multiplication circuitry attached directly to those cells, which multiplies each stored weight by a piece of the input data
• Multiple adder trees (circuits that sum up partial products in parallel) that handle the accumulation step
• A multiplexer module (essentially a configurable routing switch) that decides which partial result feeds into which adder tree, selecting one wiring pattern for normal VMM and a different pattern for transposed VMM

In a conventional approach, performing the transposed version requires either physically rearranging the data in memory or loading a second, pre-transposed copy. This design avoids both by making the routing logic reconfigurable while the weights stay stationary.

What this means for the cost of running AI models

AI inference and training are dominated by matrix operations. Every token your chatbot generates, every image a diffusion model renders, involves thousands of these multiplications per second. Hardware that cuts the overhead of switching between matrix orientations could reduce memory bandwidth pressure, which is one of the primary bottlenecks on today's AI accelerators.

For OpenAI specifically, which operates at enormous scale, even incremental gains in per-operation efficiency compound into real cost savings. This patent also signals that OpenAI is actively developing its own custom silicon thinking, a space currently dominated by Nvidia, AMD, and Google's TPUs. Whether this specific design ever ships in a production chip is an open question, but the IP establishes a foothold.

The drawings

6 drawing sheets from US 2026/0220227 A1 · click any drawing to enlarge

SHEET 1 SHEET 2 SHEET 3 SHEET 4 SHEET 5 SHEET 6

Browse all the drawings 6 drawing sheets

× ← →

Editorial commentary on a publicly published patent application. Not legal advice.