AMD Patents Speed Boost for AI Chips That Process Training Data Faster
Reading data from the wrong direction is one of those quiet performance killers inside AI chips. AMD's new patent tackles it by teaching the chip to keep a pre-flipped copy of the data on standby.
What AMD's transposed matrix trick actually does
Imagine a spreadsheet. Reading across a row is fast and natural. But reading down a column is awkward because computers store data row by row, so grabbing a column means skipping around in memory over and over.
AI chips do enormous amounts of math on grids of numbers called matrices, and they constantly need to read both rows and columns. AMD's patent describes a way to handle that column-reading problem by having the chip automatically create a "flipped" copy of the matrix, where the rows and columns are swapped. Reading a column from the original then becomes reading a row from the flipped copy, which is fast.
The clever part is that the chip checks whether a flipped copy already exists before making a new one. If it does, it reuses it. If it doesn't, it generates one on the spot. No software changes required.
How the AU generates and reads the transposed clone
The patent describes an accelerator unit (AU) built around special high-speed storage slots called tile registers. These registers hold the matrices (grids of numbers) that an AI model works with during computation.
The core feature is a new instruction called a tile read column instruction. When the chip needs to pull a specific column out of a stored matrix, it follows this process:
- Check whether a transposed clone (a version of the matrix with rows and columns swapped) is already sitting in the tile registers.
- If no transposed clone exists, the unit generates one using a sequence of internal row-read and tile-write commands.
- Once the transposed clone is confirmed to be present, the chip reads the rows of that clone that correspond to the requested column in the original matrix.
The underlying insight is that modern chip memory is optimized for sequential row access. Column access in a row-major layout (the standard way data is stored) is inherently slower because each element is physically separated in memory. By maintaining a transposed copy, every column lookup becomes a row lookup, which the hardware handles efficiently.
What this means for AMD's AI chip ambitions
AI training and inference workloads are dominated by matrix multiplication. The way data is laid out in memory can have a significant effect on how fast those multiplications run, especially when the chip needs to access data in a non-standard direction. This kind of low-level hardware optimization is exactly where companies like AMD and Nvidia compete when raw core counts start to look similar on paper.
For AMD's Instinct line of data-center AI accelerators, patents like this suggest active work on the instruction-set architecture that sits beneath software like ROCm. If this technique ships, application developers wouldn't need to do anything differently; the speed benefit would arrive at the hardware level.
This is a genuinely useful piece of chip engineering, not a flashy AI headline. The problem it solves (slow column reads in matrix hardware) is real and well-known, and the solution is clean. It won't get AMD on the evening news, but it's the kind of incremental work that compounds into meaningful performance gains over a chip generation.
The drawings
6 drawing sheets from US 2026/0219942 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.