OpenAI Patents a Chip That Handles Precise Calculations Without Slowing Down
OpenAI has filed a patent for a chip architecture that can handle different types of math, decimal-style and whole-number, inside the same block of memory, without shipping data back and forth between separate components.
What OpenAI's mixed-math memory chip actually does
Ever tried to multitask between two totally different kinds of work using the same desk? That's the problem AI chips face: different calculations need different math formats, and switching between them wastes time and energy.
OpenAI's patent describes a compute engine that can flip between floating-point math (the kind with decimals, used for high-accuracy work) and integer math (simpler whole numbers, used for speed) without changing hardware. The engine lives inside or alongside the memory itself, so numbers don't have to travel far before being crunched. The result is less wasted movement of data, which is one of the biggest drains on AI processing today.
The design also handles different levels of precision, think of precision like the number of digits after a decimal point, and automatically scales and adjusts results so nothing gets rounded off badly. It's aimed at making AI inference (running a trained model) faster and cheaper on specialized chips.
… format preprocessing circuitry arranged along input paths and coupled between the operand storage and at least one of the compute circuitry and the accumulation circuitry, and configured to route element representations of different bit-widths through shared lanes; …
Translation: Special hardware routes different sizes of data through the same pathways to save space.
How the compute engine handles floating-point inside memory
The patent describes a compute-in-memory (CIM) architecture, meaning the math happens as close to where the numbers are stored as possible. Normally, a chip reads data from memory, ships it to a separate math unit, computes, then writes results back. Each trip costs energy and time. CIM cuts that round trip.
At the center is a configurable compute engine that can operate in two modes:
- Floating-point mode: handles numbers with fractional parts (like 3.14159), needed for high-accuracy calculations in AI models.
- Integer mode: handles whole numbers, which are faster and cheaper to compute but less precise.
The engine includes format preprocessing circuitry, circuits that take numbers of different bit-widths (think: different size containers for numbers) and route them through the same shared hardware lanes, so one set of circuits handles many formats without duplicating silicon.
For floating-point work, it separately tracks the mantissa (the significant digits) and the exponent (the scale), then uses a power-of-two alignment network to line numbers up correctly before adding them. Parallel adder trees (multiple addition circuits running at once) feed into a hierarchical accumulator that collects partial sums into a final result. A normalization step then uses a slightly wider internal precision to clean up rounding errors, and applies block-level scaling with dequantization (converting compressed numbers back to their real-world scale) to keep results accurate even when using low-precision inputs.
Normalization operates at a wider intermediate precision, applies block-level scaling with dequantization, and, in lower-precision operation, performs groupwise normalization with an intermediate-precision combine to preserve a constant compute-block size.
Translation: The chip maintains high precision during calculations by scaling and normalizing data in stages.
What this means for AI chip efficiency and cost
Energy and memory bandwidth are the two biggest cost drivers in running large AI models. Moving data between memory and compute units is expensive at scale. A chip that does math inside the memory block directly cuts that cost. For companies running inference at massive scale, serving millions of AI requests per day, even small efficiency gains per operation compound into serious savings.
The flexibility to switch between floating-point and integer math in one engine also matters because modern AI workloads mix both: some layers of a neural network need high precision, others can get away with rougher integers. Having one configurable engine handle both, rather than separate dedicated circuits, could reduce chip area and manufacturing cost. AI chip design is one of the most actively contested areas in the latest Big Tech patents, and OpenAI staking out compute-in-memory territory signals the company is thinking carefully about owning more of its own silicon stack.
OpenAI's third filing we've tracked since July in the AI chip wars, after one that switches math types and one for uninterrupted AI math, applies for another chip patent.
Moving data back and forth between storage and the processor eats up a huge slice of what it costs to run AI at scale. That cost grows as AI models get bigger and run more often on more devices.
This design tackles the problem at its root by putting the number-crunching power right next to the memory, instead of patching things up after the fact. The engineering covers all the messy real-world details that separate a clever idea from something you can actually ship.
The fix is as big as the problem. That matters.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0244402 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →