Upscaling textures for VFX and games using AI
Textures define the look of the surfaces for our characters, assets, and environments in any 3D endeavour. These textures are created in texturing programs and are themselves created by layering other textures together to get the desired look. Especially in VFX, we use a wide variety of different textures to create our desired looks. It is not uncommon for a 3D asset or character to be comprised of hundreds if not thousands of different textures, both in the texturing process and the final PBR maps.
Sometimes, the textures that we use are low resolution and this can cause issues for the final output as what you are working with isn’t as sharp as it should be. Usually if you run into this problem with your source textures you would need to find and use a different texture that is of a higher resolution. Now, with AI, we can upscale these blurry and low resolution textures to make them very sharp.
In this post, I explore using AI to upscale source textures, like masks, for use in the texturing process as well as upscaling full PBR maps to make the assets a higher resolution. There are a lot of tools that you can use to upscale images, I opted to use Topaz Labs GigaPixel AI as I have found it is a very solid product with great flexibility.
The following texture comparisons are best viewed on a desktop PC.
Images on the left are default (4K) and the images on the right are AI upscaled.
Mask textures
Below I’ve upscaled 4K mask textures to see how much extra detail we can recover from them. The comparison images are zoomed in on the 4K image for better clarity in reviewing the difference between them. The upscaled resolution is 16K.
PBR textures
Here I wanted to test how well the upscaling would work when used on a full set of PBR textures from the Megscans library. I upscaled all of the PBR maps for the asset: Diffuse, Roughness, Normal, and Displacement. The original textures are 4K. I upscaled them to 16K for these comparisons to see how far I could push the upscaling.
Native 8K textures vs Upscaled 8K textures
Below I tested how well the upscaling holds up compared to native 8K textures. I upscaled the 4K textures to 8K and compared it to the native 8K textures provided by Megascans.
Verdict
As we can see from the above comparisons, AI can be used to great effect to upscale source textures to use in the texturing process as well as full PBR textures. This allows us as artists to not be locked into a set resolution for our textures and gives us the ability to upscale to a much higher resolution.
In the near future, it will be possible to do these upscaling of textures in the rendering engine as it is required. Currently, rendering engines mip-map the provided textures to lower resolutions to save on performance when the full resolution texture is not needed. However, they do not generate higher resolution textures for when the full resolution texture provided isn’t enough. Nvidia has published research addressing this exact issue, allowing for very low size high resolution textures to be used even in real time renderers such as Unreal Engine. This will allow us to provide a single texture and let the rendering engines deal with the up and down scaling as it is required based on the textures visibility in the render with very little quality lost in the process.
