IBM Patents Technology That Runs Massive AI Brain Across Multiple Connected Processors

By Patentlyze Team · Updated Jul 31, 2026

Running a large AI model usually means sending data back and forth to external memory, which is slow and power-hungry. IBM's new patent describes a way to slice the model up so each chip in a chain holds its own piece, and data just flows forward through them like an assembly line.

See all 13 drawings from this filing ↓

What IBM's on-chip LLM partitioning actually does

Imagine a factory assembly line where each station does one specific job and then passes the work to the next station, rather than one giant machine doing everything at once. That's roughly what IBM is patenting here for AI models.

A large language model (the kind of AI that powers chatbots) normally has many layers of computation stacked on top of each other. IBM's approach cuts those layers into slices, loads each slice onto a separate chip, and lets the output of one chip flow directly into the next. Each chip stores its piece of the model in its own built-in memory, so there's no constant back-and-forth with a central memory bank.

The result is that a very large AI model could run across a sequence of smaller, specialized chips rather than requiring one enormous, expensive processor. That's potentially useful anywhere you want AI to run locally, from data center appliances to dedicated inference hardware.

How each chip stores and passes AI activations forward

The patent describes a method for running LLM inference (the process of generating an answer from a trained AI model) across a sequence of chips using what it calls activation forwarding.

A typical large language model is built from three types of layers stacked in order:

• An embedding layer, which converts input tokens (words or word-pieces) into numerical vectors.
• Transformer layers, the main computational workhorse, which process and refine those vectors repeatedly.
• An output layer, which converts the final vectors into a predicted next token.

IBM's method partitions the transformer layers so that each partition (one or more transformer layers) lives on a single chip, with the trained model weights stored in that chip's on-chip memory. When a layer finishes computing, it passes its output, called an activation (essentially the intermediate result), directly to the next chip in the sequence, which handles the next partition.

Because each chip holds its own weights locally, the system avoids repeatedly fetching data from off-chip memory, which is one of the main bottlenecks in modern AI inference hardware. The final chip in the chain produces the model's actual output.

What this means for running AI without giant server farms

The main cost of running large AI models today is not raw computation but memory bandwidth, the time spent shuttling weights back and forth between a processor and external memory. By pinning each model slice to a chip that permanently holds those weights, IBM's approach could cut that bottleneck significantly for dedicated inference hardware.

This kind of architecture is particularly relevant for neuromorphic or custom AI chips, an area IBM has invested in heavily. If inference can be distributed across a chain of smaller chips, it opens a path to building AI systems that are more energy-efficient and easier to scale by just adding more chips to the chain, rather than upgrading to a single larger processor.

The drawings

13 drawing sheets from US 2026/0220446 A1 · click any drawing to enlarge

SHEET 1 SHEET 2 SHEET 3 SHEET 4 SHEET 5 SHEET 6 SHEET 7 SHEET 8

Browse all the drawings 13 drawing sheets

× ← →

Editorial commentary on a publicly published patent application. Not legal advice.