Get the newsletter
Groq · Chip · Filed Jul 20, 2026 · USPTO published Nov 19, 2026 · This week's release

Groq Files Patent for Cutting Giant AI Models Into Pieces Across Many Chips

Groq's chips hold so little memory that one big AI model can need hundreds of them. This patent describes the software that carves the model up and hands each chip its share.

An eight-chip network designed to share large artificial intelligence computing tasks through direct connections between chips. Drawing from patent filing US 2026/0367992 A1.
An eight-chip network designed to share large artificial intelligence computing tasks through direct connections between chips.All 21 drawings ↓
Publication number
US 2026/0367992 A1(US20260367992A1)
Applicant
Groq LLC
Filed · Published
Jul 20, 2026 · Nov 19, 2026
Inventors
Kyeong Mo Kang, Yuxi Cai, Naif Tarafdar, Andrew Chaang Ling, Pao-Sheng Chou
Status when we published
Examiner raised objections; the company can respond
Document
20 claims
US classification
718/104
Examiner
KIM, DONG U (Art Unit 2197)
Parent application
Continuation of 18/325,647 (filed May 30, 2023)(18325647)

What Groq's model-slicing software actually does

Imagine you own a sofa that won't fit through your front door. You can't shrink it, so you take it apart, carry it in pieces and rebuild it inside. Giant AI models have the same problem with computer chips: many are too big for the memory on a single chip.

Groq's patent describes software that does the taking-apart for you. It cuts a model in two ways. One way hands whole steps of the model to different chips, like giving each mover a different piece of the sofa. The other way slices a single heavy step in half so two chips share the load, like two people lifting one cushion that's too big for one.

The software does both kinds of cutting in one pass. It checks that no chip is asked to hold more than it can handle, then prepares the instructions each chip will follow.

From the filing · CLAIM 1
performing at least one horizontal slicing operation configured to assign a first computational operation of the machine learning model to a first partition and a second computational operation of the machine learning model to a second partition …

Translation: Splitting the AI model horizontally assigns different computational tasks to separate partitions.

How Groq cuts one AI model across many chips

The patent covers a compiler (software that translates an AI model into step-by-step instructions a chip can run). It splits a machine learning model into partitions, then compiles each partition for its own processor. The claims describe two kinds of cuts:

  • Horizontal slicing: assigns whole operations (a tensor operation is a big math step on grids of numbers) to different chips, keeping each operation intact.
  • Vertical slicing: divides a single operation so two or more chips each do part of it.

The description gives concrete numbers. Groq's chip has 720,856 memory addresses, and one example model needs 751,000, so it is split into 500,000 and 251,000. In another example, a high-definition image step needing over 1.048 million addresses is cut in half, 524,288 per chip.

The claims say both cuts can happen in a single pass, based on each step's resource needs, to cap peak live state (the most memory a chip must hold at one moment). The chips run on deterministic timing, meaning the compiler fixes when every operation happens ahead of time. The cuts can also account for how the chips are wired, including network bandwidth and the number of hops between them. The filing also describes grouping runs of simple operations into clusters.

Why splitting models matters for Groq's chips

The practical stake is simple: models too big for one chip can still run. The filing says the approach lets chips with small local memory handle language models like ChatGPT or Llama, with low delay and high throughput. For you, that could mean chatbot answers that arrive faster and cost less to serve, though the patent makes no product promises.

It also shows how Groq thinks about its hardware. The chips depend on the compiler planning everything in advance, so software like this is what makes hundreds of chips behave like one machine. The application is a continuation of a 2023 filing, and the examiner's first review rejected the claims as filed, which is routine at this stage and usually followed by amendments.

Groq's second patent in our chip filings we cover since November follows one on pre-planned data routing.

Source: US 2026/0367992 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Groq patent news breaks.

There are more where this came from

We read the new patents from Big Tech, carmakers and more and send you the ones worth knowing. Plain English, free, every week.

The drawings · 21 sheets

21 drawing sheets from US 2026/0367992 A1 · click any drawing to enlarge

Translating user code into instructions for an AI chip, drawing sheet 1 from US 2026/0367992 A1
Sheet 1 Translating user code into instructions for an AI chip
Steps that prepare an AI program for execution, drawing sheet 2 from US 2026/0367992 A1
Sheet 2 Steps that prepare an AI program for execution
Splitting a single math task across two chips, drawing sheet 3 from US 2026/0367992 A1
Sheet 3 Splitting a single math task across two chips
Dividing an AI program vertically or horizontally, drawing sheet 4 from US 2026/0367992 A1
Sheet 4 Dividing an AI program vertically or horizontally
Counting the memory addresses an uncut program needs, drawing sheet 5 from US 2026/0367992 A1
Sheet 5 Counting the memory addresses an uncut program needs
Cutting a program in half to fit two chips, drawing sheet 6 from US 2026/0367992 A1
Sheet 6 Cutting a program in half to fit two chips
Large image data before being split up, drawing sheet 7 from US 2026/0367992 A1
Sheet 7 Large image data before being split up
Halving image size to fit work across two chips, drawing sheet 8 from US 2026/0367992 A1
Sheet 8 Halving image size to fit work across two chips
Show the other 13 sheets
Combining cuts to lower memory demand across four chips, drawing sheet 9 from US 2026/0367992 A1
Sheet 9 Combining cuts to lower memory demand across four chips
Grouping related math tasks into simpler clusters, drawing sheet 10 from US 2026/0367992 A1
Sheet 10 Grouping related math tasks into simpler clusters
Slicing giant matrix calculations to fit separate chips, drawing sheet 11 from US 2026/0367992 A1
Sheet 11 Slicing giant matrix calculations to fit separate chips
Stepwise translation of an AI model into chip instructions, drawing sheet 12 from US 2026/0367992 A1
Sheet 12 Stepwise translation of an AI model into chip instructions
Passing finished calculations from one chip to another, drawing sheet 13 from US 2026/0367992 A1
Sheet 13 Passing finished calculations from one chip to another
Splitting a single calculation halfway across two chips, drawing sheet 14 from US 2026/0367992 A1
Sheet 14 Splitting a single calculation halfway across two chips
Direct network links connecting eight AI chips, drawing sheet 15 from US 2026/0367992 A1
Sheet 15 Direct network links connecting eight AI chips
Wiring four chips together inside a high-power server rack, drawing sheet 16 from US 2026/0367992 A1
Sheet 16 Wiring four chips together inside a high-power server rack
Speed and delay test results running AI across chip racks, drawing sheet 17 from US 2026/0367992 A1
Sheet 17 Speed and delay test results running AI across chip racks
How instructions and data move across a specialized AI chip, drawing sheet 18 from US 2026/0367992 A1
Sheet 18 How instructions and data move across a specialized AI chip
Inside layout of memory and calculation zones on the chip, drawing sheet 19 from US 2026/0367992 A1
Sheet 19 Inside layout of memory and calculation zones on the chip
East and west data paths running through a single row, drawing sheet 20 from US 2026/0367992 A1
Sheet 20 East and west data paths running through a single row
Standard computer parts that host and run the compiler system, drawing sheet 21 from US 2026/0367992 A1
Sheet 21 Standard computer parts that host and run the compiler system
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.