Samsung Patents a Way to Collapse AI Processing Steps Onto a Single Chip
Every time an AI model hands data from one processing step to the next, memory gets touched and time gets wasted. Samsung's new patent describes a way to bundle those steps together so the chip never has to leave its fastest, smallest memory to get the job done.
What Samsung's layer-fusion method actually does for AI chips
Imagine a factory assembly line where every worker passes a partially finished part to a bin, and the next worker has to walk over, pick it up, and carry it to their station. That back-and-forth adds time even if the actual work at each station is fast. AI chips face the same problem: every "layer" of a neural network (each mini-step of the computation) produces data that has to be stored somewhere before the next step reads it.
Samsung's patent describes a smarter way to schedule that work. Instead of treating each layer as a separate stop on the line, the system figures out which layers can be fused together into one single operation that runs entirely inside the chip's fast on-chip memory, skipping the slow trip out to main memory.
The trick is identifying specific structural patterns inside the AI model (called residual blocks, a common building block in image-recognition and other AI tasks) and finding the right "boundary" layer where a big batch of preceding steps can all be collapsed together. The result is a leaner execution plan that should let the hardware do more work per second while burning less energy.
… determining an outermost merge layer comprised in each of the plurality of residual blocks; determining at least one fusion receiving layer from among a plurality of outermost merge layers of the plurality of residual blocks; …
Translation: The system finds specific connection points where different processing steps can be safely joined together.
How the patent picks which AI layers to merge and when
Neural networks are organized into layers: each layer transforms data in some way and passes the result to the next. On real hardware, moving that data between layers usually means writing it to memory and reading it back, which is slow. Layer fusion is the technique of merging consecutive layers so their combined computation happens in one pass, keeping intermediate results in the chip's tiny, fast on-chip memory (called SRAM or a scratchpad) rather than shipping them out to slower DRAM.
The challenge is that modern AI models are not simple chains; they branch and reconnect. A residual block (a structure popularized by the ResNet family of image models) splits the data flow into two paths that later merge back together at an "add" or "concatenation" step. That merge point is what Samsung calls the outermost merge layer, and finding it is the first key step in the patent's method.
Once the system maps all those merge points across the full model, it picks certain ones to act as fusion receiving layers: the designated endpoints where a large batch of earlier layers can be collapsed. Everything on the incoming path to that endpoint becomes one fused layer group, and the whole group is compiled into a single on-chip execution unit.
- Parse the AI model's operation graph to locate all residual blocks.
- Find the outermost merge layer in each block (the point where split paths rejoin).
- Select which merge layers serve as fusion boundaries.
- Group all upstream layers into a fused set and generate one combined execution plan.
… generating a fused execution graph corresponding to a first single on-chip operation unit by performing the layer fusion on the plurality of fused layers.
Translation: Multiple layers of AI calculations are combined so they can run all at once on a single chip component.
What this means for AI running on Samsung's own hardware
For Samsung, which designs its own AI accelerator chips (the Exynos NPU and Mobicore family), a patent like this is less about theory and more about keeping up with rivals like Qualcomm and Apple, who have their own compiler and fusion optimizations baked into their chip toolchains. Samsung's steady investment in on-device AI compiler work suggests the company wants its chips to run large AI models locally, without relying on cloud servers, which matters a lot for privacy and speed in mobile devices.
For you as a user, the practical payoff would be AI features (photo editing, voice recognition, real-time translation) that run faster and drain your battery less. The filing targets the compiler layer, which is invisible to end users but determines how efficiently the hardware underneath your app actually runs.
That makes this Samsung's 48th filing we've tracked since June in the AI chip wars watch, following one on compressing AI and one on unpacking AI data.
The tradeoff at the center of this design is specificity versus generality. The method is built around residual blocks, a structure that is everywhere in image-processing models but far less dominant in transformer-based models (the kind powering large language models and most modern generative AI). If Samsung's hardware future tilts toward transformers, a fusion strategy tuned to residual-block boundaries could end up as scaffolding for yesterday's architecture.
That said, the actual cost of that bet is probably low. Residual blocks are not going away; they are deeply embedded in computer vision and many on-device inference tasks that Samsung's mobile chips handle daily. A compiler optimization that works reliably on that class of models is worth shipping even if it does not cover every case.
The bigger open question is how much the fusion boundary selection is automated versus manually tuned. The patent describes the decision logic at a high level but does not reveal how the system chooses among candidate merge layers when multiple valid options exist. That selection policy is where the real performance delta lives, and its absence here means the patent protects the structure of the idea more than any particular quality of implementation.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
12 drawing sheets from US 2026/0268146 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →