Sony Files Patent for Turning a Photo Into an Editable 3D Game Object
Sony Interactive Entertainment has filed to turn a photo, or a sentence like 'brown chair with a bannister back,' into a 3D object a game can use. The claims focus on how an edit gets blended into the object so the seam disappears.
What Sony's photo-to-3D game asset patent does
Ever tried to describe a chair over the phone, down to the brown cushion and the curved back, and hoped the other person pictured it right? Game designers face a version of that all day, except someone then has to build every chair, table and lamp in 3D by hand.
Sony's patent application describes a tool that starts with a picture. It can be a photo you upload, or an image the software draws from your typed or spoken description. The software then builds a 3D version of that picture that can be dropped into a game.
Want a change? Ask for it, and the software blends the edited part into the rest of the object so you don't see a patchy seam. The filing also describes asking for a whole scene, such as a couch to the left and in front of a fancy chair, and checking that furniture won't tip over.
… modify the 3 D asset based on the input and by varying weights of interpolated pixels along at least one border region in the 3 D asset at which a modification is identified; …
Translation: It adjusts specific pixels around the edges when you change the object.
How Sony blends edits into a 3D object's seams
At its core, claim 1 describes software that generates a 3D asset from a 2D image, accepts an input asking for a change, and shows the modified asset in a computer simulation such as a game. An 'asset' here is a piece of game scenery, like a chair or a couch.
The edit step is the specific part. The software modifies the asset by varying the weights of interpolated pixels along the border region where the change happens. Put simply, it blends the old and new parts gradually across the seam so the edit doesn't look pasted on.
The description fills in how the rest could work:
- A spoken or typed description, such as 'brown chair with arms, 4 legs, cushioned surface and bannister back,' is broken into keywords (object, color, legs, surface, back).
- A neural network (software trained on many examples) turns those keywords into a 2D image, or the user simply uploads a photo.
- An encoder-decoder model (one half squeezes the picture into numbers, the other half rebuilds a 3D mesh, the wireframe skin of an object) produces the 3D shape.
The dependent claims add attaching audio based on text tied to the image, handling a described location or several objects, and running physics modeling of environmental effects. The description gives a tipping example: a physics engine checks whether furniture falls over when pushed, and the dimensions and weights of its parts are adjusted so it stays upright.
A 3 D scene is generated consisting of one or more objects from a natural language description that may consist of text or voice.
Translation: You can build a virtual 3D scene just by typing or speaking a description.
What this means for game artists and designers
If you play games, you see this kind of work everywhere: the chairs, rugs and clutter that make a room feel lived in. Someone has to model each one. Sony's filing says creators could describe an asset in plain language and get a starting version that artists then refine, which could trim the grunt work of filling out a scene.
For you as a player, the payoff would be indirect: richer-looking worlds, or smaller teams building more of them. The filing is clear that humans stay in the loop. It describes artists adjusting the result by voice or point-and-click, and it frames the output as a prototype to iterate on rather than a finished product.
Sony's 24th filing in the AI simulation space we've tracked since May adds to a run that includes matching 3D characters from flat photos and one cutting lag in cloud games.
Claim 1 never mentions voice. The actual legal protection covers taking a 2D image, building a 3D object from it, accepting an edit instruction, and then blending the edited area smoothly into the surrounding object so the change fades in rather than appearing as a hard cut.
That blending step is where the claim earns its shape. Any artist or studio building 3D objects from photographs and offering edit tools would need to handle that boundary differently to stay clear of this patent.
The voice features that make the title compelling show up only in later, narrower claims, meaning the broadest protection and the most attention-grabbing capability are pulling in opposite directions. What Sony is actually locking down here is a specific editing behavior in a common game-art workflow, which is a quieter and more practical kind of protection than the headline suggests.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Sony patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0349651 A1 · click any drawing to enlarge
Show the other 3 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in