Groq Files Patent for Cutting Giant AI Models Into Pieces Across Many Chips
Groq's chips hold so little memory that one big AI model can need hundreds of them. This patent describes the software that carves the model up and hands each chip its share.

- Publication number
- US 2026/0367992 A1(US20260367992A1)
- Applicant
- Groq LLC
- Filed · Published
- Jul 20, 2026 · Nov 19, 2026
- Inventors
- Kyeong Mo Kang, Yuxi Cai, Naif Tarafdar, Andrew Chaang Ling, Pao-Sheng Chou
- Status when we published
- Examiner raised objections; the company can respond
- Document
- 20 claims
- US classification
- 718/104
- Examiner
- KIM, DONG U (Art Unit 2197)
- Parent application
- Continuation of 18/325,647 (filed May 30, 2023)(18325647)
What Groq's model-slicing software actually does
Imagine you own a sofa that won't fit through your front door. You can't shrink it, so you take it apart, carry it in pieces and rebuild it inside. Giant AI models have the same problem with computer chips: many are too big for the memory on a single chip.
Groq's patent describes software that does the taking-apart for you. It cuts a model in two ways. One way hands whole steps of the model to different chips, like giving each mover a different piece of the sofa. The other way slices a single heavy step in half so two chips share the load, like two people lifting one cushion that's too big for one.
The software does both kinds of cutting in one pass. It checks that no chip is asked to hold more than it can handle, then prepares the instructions each chip will follow.
performing at least one horizontal slicing operation configured to assign a first computational operation of the machine learning model to a first partition and a second computational operation of the machine learning model to a second partition …
Translation: Splitting the AI model horizontally assigns different computational tasks to separate partitions.
How Groq cuts one AI model across many chips
The patent covers a compiler (software that translates an AI model into step-by-step instructions a chip can run). It splits a machine learning model into partitions, then compiles each partition for its own processor. The claims describe two kinds of cuts:
- Horizontal slicing: assigns whole operations (a tensor operation is a big math step on grids of numbers) to different chips, keeping each operation intact.
- Vertical slicing: divides a single operation so two or more chips each do part of it.
The description gives concrete numbers. Groq's chip has 720,856 memory addresses, and one example model needs 751,000, so it is split into 500,000 and 251,000. In another example, a high-definition image step needing over 1.048 million addresses is cut in half, 524,288 per chip.
The claims say both cuts can happen in a single pass, based on each step's resource needs, to cap peak live state (the most memory a chip must hold at one moment). The chips run on deterministic timing, meaning the compiler fixes when every operation happens ahead of time. The cuts can also account for how the chips are wired, including network bandwidth and the number of hops between them. The filing also describes grouping runs of simple operations into clusters.
Why splitting models matters for Groq's chips
The practical stake is simple: models too big for one chip can still run. The filing says the approach lets chips with small local memory handle language models like ChatGPT or Llama, with low delay and high throughput. For you, that could mean chatbot answers that arrive faster and cost less to serve, though the patent makes no product promises.
It also shows how Groq thinks about its hardware. The chips depend on the compiler planning everything in advance, so software like this is what makes hundreds of chips behave like one machine. The application is a continuation of a 2023 filing, and the examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments.
Groq's second patent in our chip filings we cover since November follows one on pre-planned data routing.
Source: US 2026/0367992 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Groq patent news breaks.
We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.
The drawings · 21 sheets
21 drawing sheets from US 2026/0367992 A1 · click any drawing to enlarge
Show the other 13 sheets

No comments yet