Sony Patents a Way to Paint Artistic Styles onto 3D Scenes Using Flat Images
Sony has filed a patent for a method that takes a 3D scene, flattens it into a series of ordinary 2D snapshots, applies an artistic style to those snapshots, then rebuilds the 3D scene in that new visual style. It sidesteps the hard problem of styling 3D data directly by turning it into a problem that existing AI tools already know how to solve.
How Sony's 3D style transfer actually works
Today, if you want a 3D model or scene to look like a Van Gogh painting or a watercolor sketch, there is no easy button. Tools that apply artistic styles work well on flat photos, but 3D data is structured very differently, and making those tools work in three dimensions requires specialized pipelines most creators don't have access to.
Sony's patent describes a system that works around that gap. Instead of trying to style the 3D scene all at once, the system photographs it from many angles, producing a stack of ordinary flat images. A standard AI style-transfer network then paints the chosen artistic look across each of those flat photos. Finally, the system reads all those newly styled flat photos and reconstructs a 3D version of the scene that now carries the new visual style throughout.
The result is that you could take a detailed 3D scan or computer-generated scene and ask the system to make it look like it was painted in a particular style, without needing specialized 3D-native AI tools to do it.
… generate a plurality of 2D projection images based on the acquired 3D volumetric data; prepare an input for a style transfer neural network based on the generated plurality of 2D projection images and the acquired 2D style image data …
Translation: The system creates flat snapshots of a 3D object and feeds them into an AI along with a reference image to learn a new style.
Inside Sony's project-style-reassemble pipeline
The system described in the patent works in four stages:
- Acquire the data: The system takes in a 3D volumetric dataset (think a LiDAR scan, a medical imaging volume, or a game-engine scene stored as a point cloud or voxel grid) and a 2D reference image that defines the desired artistic style.
- Project to 2D: It renders multiple flat photographs of the 3D scene from different viewpoints, called projection images. Each one is just a normal picture, like screenshots taken from different camera angles.
- Style transfer: Those flat images, paired with the style reference, are fed into a style transfer neural network (an AI model trained to copy the visual texture, color, and brushstroke character of one image onto another). The network produces a stylized version of each flat photograph.
- Reconstruct in 3D: The system then uses the collection of stylized flat images to rebuild a 3D volumetric dataset that now carries the applied style in three dimensions.
The key insight is that reusing existing 2D style-transfer AI avoids building an entirely new 3D-native model from scratch. The reconstruction step is where the technical complexity sits: the system has to figure out how to merge style information from multiple flat views into a coherent 3D output without visual conflicts between angles.
The system generates a plurality of stylized 2D projection images based on application of the style transfer neural network on the prepared input and obtains a stylized 3D volumetric data based on the plurality of stylized 2D projection images.
Translation: After the AI applies the artistic style to the flat snapshots, the system stitches them back together into a 3D model.
What this means for 3D content and game visuals
For anyone creating 3D content, whether in games, film, AR, or medical visualization, this kind of tool could make artistic restyling far more accessible. Right now, applying a consistent visual style across an entire 3D scene is either a manual art job or requires bespoke AI infrastructure that most studios don't have. A pipeline like this could slot into existing 3D content workflows without requiring new specialized hardware.
Sony has obvious surface area here: PlayStation, its film and TV studios, and its professional camera and imaging divisions all touch 3D content in some form. The latest Big Tech patents in the 3D graphics and volumetric media space show a clear push from major players to close the gap between 2D AI tools and 3D production pipelines, and this filing is a concrete step in that direction.
The ship path here is shorter than most AI graphics patents because the system explicitly relies on existing 2D style-transfer networks rather than proposing new model architectures. The novel engineering work is in the projection and reconstruction steps, both of which are tractable software problems Sony's imaging and codec teams already have context on. The biggest open question the patent leaves unanswered is how well the multi-view reconstruction handles style inconsistencies across angles, which is the part most likely to require additional tuning before any product can use it reliably.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
11 drawing sheets from US 2026/0237148 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →