Sony Files Patent to Give Synthetic Voices Separate Controls for Accent and Mood
Sony's patent application describes an AI voice that can copy someone's sound, then change the mood or accent without touching the rest. The trick is breaking one voice recording into separate labeled pieces.
What Sony's voice-cloning patent actually does
A navigation app reads out 'Turn left now' in the same mild tone whether you are relaxed or about to miss your exit. Today's synthetic voices can copy a person's sound, but they often struggle to also copy how that person speaks: the mood, the accent, the pace.
Sony's patent application goes after this by splitting a voice into separate parts. Instead of handing the AI one audio clip and hoping it figures out everything, the system gives it separate hints for who is talking, how the voice sounds, how emotional it is, what accent it has, how long the speech should run, and what language it is in.
The idea is that each hint has one job, so the AI learns its lessons more cleanly and you get more control over the result. Sony says the model can be trained on about 7,000 hours of recordings, where ordinary versions of this technology need at least 60,000.
… performing audio reference voice embeddings for each voice attribute using a divide and conquer mechanism to generate audio reference tokens; training the LLM to recognize what task each audio reference voice embedding belongs using engineered features; …
Translation: The system breaks down different voice traits into separate parts so the AI can handle them individually.
How Sony splits a voice into separate parts
The patent describes a text-to-speech system built on a large language model (the same family of AI behind chatbots), trained here to produce speech instead of text. It is zero-shot, meaning one trained model can imitate a voice it never heard during training, given a short reference recording.
The twist is a divide and conquer approach to that reference audio. Rather than squeeze everything into one summary of the clip (a mel-spectrogram, which is basically a picture of the sound), the system builds separate embeddings (lists of numbers that stand for one trait) for each trait:
- Identity: a speaker embedding built from speaker verification features.
- Voice quality: the sound spectrum of the clip.
- Expressivity: pitch (called F0), energy and emotion recognition features.
- Accent: accent recognition features.
- Duration: a dedicated embedding for how long the speech should last.
Language is the sixth trait the claims list. Two training penalties keep things tidy. A consistency loss (a penalty when the output drifts from the target spectrum, pitch, energy or text) improves sound and pronunciation. An orthogonality loss (a penalty when two embeddings carry overlapping information, checked with cosine similarity) keeps, for example, the accent summary from leaking into the speaker summary.
… smaller but more accurate space to learn for each embedding instead of one mel-spectrogram embedding for all possible attributes.
Translation: Breaking down the audio data makes the AI's job simpler and more precise than processing everything at once.
Why Sony's smaller training budget matters
For you, the payoff would be synthetic voices you can steer: keep the same speaker but change the mood, or keep the mood but change the accent. That matters anywhere a device reads text aloud, which the filing notes is increasingly common on mobile devices. Sony also says the approach needs far less training audio, about 7,000 hours against 60,000 or more, which would lower the cost of building voices for many accents and languages.
It is early days. The examiner's first review rejected the claims as filed, which is routine and usually followed by amendments. The filing also offers no listening tests, so how natural the results sound is unknown.
This is the sixth Sony filing we've tracked in voice and speech AI since May, following one on picking a narration voice by topic and one on reading your voice for multiplayer matches.
Splitting a voice into labeled pieces costs something. The system needs a separate tool to measure each piece, such as one that recognizes who is speaking, one that reads emotion and one that spots accents. If any of those tools makes a mistake, the voice inherits that mistake.
There is a second risk. Real speech blends mood, accent and pace together, and forcing the pieces apart with extra training rules could flatten the natural mix that makes a voice sound alive. The filing pushes the pieces apart on purpose, so a happy tone does not leak into the accent, and it never says how much naturalness that discipline might cost.
Still, the trade looks worth making on paper. Needing about 7,000 hours of training audio against 60,000 is a big saving, and control over mood and accent is what a narrator or assistant voice needs. The 7,000 figure is Sony's own claim, with no test results in the text.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
6 drawing sheets from US 2026/0341264 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in