Intel Patents a Way to Keep AI Chips Fast When Data Sizes Keep Changing
AI chips are built for speed, but they hate surprises. Intel is patenting a way to keep deep learning hardware from stumbling every time the data it's processing arrives in an unexpected shape or size.
What Intel's dynamic-shape AI chip fix actually does
Every time an AI model runs on your device or in the cloud, it chops data into chunks and sends them through a chip. When those chunks are always the same size, the chip handles them efficiently. The problem is that real-world AI tasks, like translating text of varying lengths or analyzing images of different resolutions, keep feeding the chip chunks of unpredictable sizes.
Intel's patent describes a system that watches the range of data sizes a workload typically throws at a chip, then pre-builds several optimized processing plans for each common size range. Instead of stumbling and recalculating every time the data changes, the chip already has a plan ready to go.
The goal is for AI hardware to stay fast and efficient even when the workload is messy and inconsistent, which describes almost every real product that uses AI today.
infer, from the profile of tensor shapes, a profile of parallel index ranges and a probability for each of the parallel index ranges; determine a set of candidate blocking factors for each of the parallel index ranges; and generate a kernel for the deep learning operation under each of the candidate blocking factors …
Translation: The system predicts how data sizes will vary and prepares optimized code variations ahead of time.
How the chip picks blocking factors for shifting workloads
The patent centers on what Intel calls a dynamic-shape workload problem. Neural networks process data organized into grids called tensors (think of a tensor as a numbered container holding data in rows, columns, and layers). The size of that container varies depending on the input, and chips that were tuned for one size can become inefficient when the size changes.
Intel's solution has three stages:
- Profile collection: The system gathers a statistical profile of the tensor shapes that a specific AI workload typically sends. It learns not just what sizes appear, but how often each size range shows up and assigns a probability to each.
- Blocking factor selection: For each size range, the system calculates a set of blocking factors (the specific way work gets divided across processor cores). Choosing the right split is what keeps chips busy rather than waiting idle.
- Kernel pre-generation: For each blocking factor candidate, the system generates a ready-to-run processing routine (called a kernel) that includes an outer loop spread across multiple cores and an inner loop handled by a single core. The right kernel is selected at runtime.
This is essentially the chip doing its homework in advance, so it doesn't have to improvise when odd-sized data arrives.
What this means for AI inference speed in real products
For anyone running AI in production, slow or inconsistent inference (the moment an AI model produces an answer) is expensive. Cloud providers pay for wasted compute cycles, and on-device AI experiences lag when a chip stalls. The more variable the input, the worse the problem gets.
Intel's approach targets this directly by treating unpredictability itself as something you can profile and plan for. If it works well in practice, it could mean faster and more consistent AI responses in applications that handle real human input, where message lengths, image sizes, and audio clips never arrive in neat uniform batches. Intel has been filing around AI inference optimization since 2023, and this fits that pattern.
This is the 40th Intel filing we've tracked in the AI chip wars since May, building on earlier applications like one on faster AI math and one on shifting AI strategies mid-task.
The problem Intel is attacking here is real and costly. AI chips are typically hand-tuned for fixed data sizes, and production AI systems are almost never that tidy. Every chatbot response, every image you upload, every voice query arrives at a different size, and every mismatch between data and chip expectations burns time and money.
The proposed fix is reasonable engineering: profile what you usually see, pre-compute your options, pick the best one fast. That's a sensible match for the problem's scale, though it depends heavily on whether the workload's shape distribution is stable enough to profile usefully. If the data is truly random with no pattern, the pre-built plans may not help much.
For most enterprise AI workloads, which do tend to cluster around common input sizes, this looks like a practical optimization rather than a speculative one. It's the kind of careful chip-level work that doesn't show up in product announcements but does show up in benchmark sheets.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
10 drawing sheets from US 2026/0278365 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in