Get the newsletter
Sony · AI / ML · Filed Jul 23, 2026 · USPTO published Nov 19, 2026 · This week's release

Sony Patents an AI Game Soundtrack That Reacts to Your Heart Rate

Sony's patent application describes a game soundtrack that an AI writes as you play, steered by readings like your heart rate, sweating and blinking. Your pulse becomes part of the score.

A game console paired with a handheld controller and a player wearing a headset to track bodily reactions. Drawing from patent filing US 2026/0364456 A1.
A game console paired with a handheld controller and a player wearing a headset to track bodily reactions.All 5 drawings ↓
Publication number
US 2026/0364456 A1(US20260364456A1)
Applicant
Sony Interactive Entertainment Inc.
Filed · Published
Jul 23, 2026 · Nov 19, 2026
Inventors
Daniel Israel Pereira Jimenez, Estefania Rodriguez Quiros, Lazaros Michailidis, Jesus Barcias, Christopher William Henderson
Status when we published
Waiting for an examiner
Document
20 claims
US classification
463/35
Parent application
Continuation of 18/443,515 (filed Feb 16, 2024)(18443515)

What Sony's heart-rate game soundtrack does

You're creeping through a dark cave in a game, heart pounding, and the music is the same loop you heard an hour ago. It has no idea you're nervous. Sony's new patent application is about fixing that gap.

Here's the idea. Your console watches what's happening and picks a few mood keywords from a list the game's developer wrote, words like 'cave,' 'hostile,' or 'friendly.' It sends those words to an AI that composes music on the fly and keeps reshaping it as the words change.

The filing also has the keywords respond to your body. Sensors in a VR headset or controller can pick up your heart rate, sweating, or blinking, so the score can shift when you tense up. If a jump scare hits, the music can lurch with it.

From the filing · CLAIM 1
… receive, from the entertainment device during processing of the game, one or more keywords selected based at least in part on physiological data of a user playing the game; and in response to receipt of a keyword of the one or more keywords, update the generation of the information indicative of the musical score based at least in part on the keyword.

Translation: The system listens to your body signals while you play and changes the music based on what it detects.

How keywords steer the AI composer mid-game

The claims cover a music generator that sits between the game and your speakers. When the console asks for a score, the generator starts producing music right away and streams it back as it goes. It can run on a cloud server, on a nearby device, or inside the console itself.

The key trick is the keywords. The console reads what the player is doing, including physiological data (body readings such as heart rate, eye movement, sweating, skin temperature and blinking), and picks matching words from a list the developer defined for that game. Each time a new keyword arrives, the generator updates the music. Sending a few words is far lighter than streaming the whole game state, and the filing says it also keeps the AI easier to train.

The filing describes several add-ons:

  • Seed values: extra inputs, such as a game title or in-game location, that make the same keywords produce music unique to that game or place.
  • Sudden events: a jump scare can trigger an abrupt change, or stop the AI so prerecorded audio plays instead.
  • Smooth transitions: the AI can be trained on keyword pairs like 'meadow' then 'cave' so the music blends between moods.
  • Feedback: players can say whether the music matched their mood, and that answer helps retrain the AI.

The output can be a MIDI file or stream, which is a set of note instructions rather than a finished recording.

From the filing · THE ABSTRACT
The one or more keywords are indicative of one or more of an action and/or a condition associated with a user playing the game and the game state.

Translation: Special labels describe what the player is doing and how the game is reacting at that exact moment.

Why a reactive soundtrack matters for horror games

For players, the pitch is a soundtrack that fits how you play, so two people in the same level could hear different music. Today's game scores mostly play prerecorded tracks in a set order, and the filing says even the real-time approaches are limited by how many sound snippets a game ships with. An AI composer has no such ceiling.

For developers, the keyword list is the control panel. They decide which moods exist and which game moments trigger them, and seed values keep a game's music distinct from other games using the same AI. The filing's examples use a PlayStation 5, a VR headset and a DualSense controller. One tradeoff: a game would be reading body signals like your heart rate, and some players will want to think about that.

This is the 177th Sony filing we've tracked in our PlayStation strategy watch since May, adding to work like the cloud connection test game and the AI play guide ghost.

Source: US 2026/0364456 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 5 sheets

5 drawing sheets from US 2026/0364456 A1 · click any drawing to enlarge

Gaming hardware paired with a VR headset and controller, drawing sheet 1 from US 2026/0364456 A1
Sheet 1 Gaming hardware paired with a VR headset and controller
Connecting a console to an AI music creator, drawing sheet 2 from US 2026/0364456 A1
Sheet 2 Connecting a console to an AI music creator
How player actions turn into adaptive background music, drawing sheet 3 from US 2026/0364456 A1
Sheet 3 How player actions turn into adaptive background music
Turning controller signals into soundtrack keywords during gameplay, drawing sheet 4 from US 2026/0364456 A1
Sheet 4 Turning controller signals into soundtrack keywords during gameplay
Live adjustments to the score as game events change, drawing sheet 5 from US 2026/0364456 A1
Sheet 5 Live adjustments to the score as game events change
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.