Creating 3D Asset Variations Through 2D Style Transfer and Generated Texture Maps

Abstract

Generating 3D object variations through style transfer models applied to their textures is an easy way for creating content for games and XR applications. Most workflows focus on either generating albedo textures only without changing the underlying surface and not touching the underlying surface or transforming the 3D object directly, which is computationally and resourceheavy. In this paper, we present an initial exploration of an in-between solution that aims to combine the style transfer of albedo textures with the generation of additional maps, such as normal, displacement, and roughness. The results show that the pipeline can generate variations based on different styles, which would enable the addition of smaller 3D-style surface features to objects without transforming their meshes. The project code and generated textures will be available at https: //github.com/IvanNik17/3D-Assets-From-2D-Style-Transfer.