New AI Patent Merges Written Words and Audio Into One Shared Language
Most AI models treat spoken audio and written text as completely separate problems. This Google patent describes a single neural network that handles both at once, using a shared vocabulary of tokens that can represent either.
How Google's shared text-and-audio AI actually works
Imagine asking your phone a question out loud and getting a spoken answer back, but the AI handling your request never had to hand off the job to separate 'speech' and 'text' systems. It just understood everything in one go.
Google's patent describes exactly that kind of unified AI. Instead of treating audio and text as different inputs that need different pipelines, the system converts both into a common set of building blocks called tokens. A token might represent a word, a syllable, or a chunk of sound. Once everything is in that shared format, a single language model handles the whole task.
The practical upside: the AI can, for example, take in your voice, reason about what you said, and produce a spoken response all within one model. No relay race between systems, fewer places for things to go wrong.
Inside Google's unified token vocabulary approach
The patent describes a system built around a unified token vocabulary, meaning a single list of discrete symbols that can represent both text (words, punctuation, sub-word pieces) and audio (small compressed chunks of sound). Think of it like a shared alphabet for two very different languages.
Here's the basic flow:
- An input arrives, which could be text, audio, or both mixed together.
- That input is broken into tokens from the shared vocabulary and then converted into embeddings (numerical representations that capture meaning and relationship) in a common mathematical space.
- A language model neural network (the same type of architecture behind large text models like GPT) processes the entire sequence and outputs new tokens.
- Those output tokens can spell out text, reconstruct audio, or both, depending on the task.
The key claim is that the model doesn't need separate encoders or decoders for speech versus text. The task description and the data to perform the task are all encoded together as one sequence, letting the model reason across modalities without switching gears.
What this means for the future of voice assistants
Voice assistants today typically chain together multiple specialized models: one to transcribe speech, one to reason about it, one to synthesize a response. Each handoff adds delay and potential error. A model that treats audio and text as one vocabulary could, in theory, do all of that in a single forward pass, making voice interactions faster and more coherent.
For Google, which operates both a major voice assistant and one of the leading large language model programs, this kind of architecture fits neatly into efforts to unify products like Google Assistant and Gemini under a single AI engine. Whether this specific patent describes something already shipping or something still in research is not clear from the filing alone.
This is genuinely interesting foundational work. Unifying audio and text into one model vocabulary is a real architectural challenge that the field has been working on for years, and Google has strong commercial incentive to get it right. That said, the lead claim being listed as canceled is a flag worth noticing: it suggests the application may still be in flux, and the granted scope could end up narrower than what's described here.
The drawings
5 drawing sheets from US 2026/0212167 A1 · click any drawing to enlarge
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.
Get one Big Tech patent every Sunday
Plain English, intelligent commentary, no hype. Free.
Editorial commentary on a publicly published patent application. Not legal advice.