This tutorial will demonstrate how to create smooth edges when generating displacement and normal maps from textures with UV islands. Ordinarily when importing a source map with islands hard edges can appear on the Normal Map. See below:
To overcome this artifact we can expand all of the island edges by removing alpha where there are no texture pixels. When the source map is loaded, the source map property “Alpha Threshold” will automatically expand the edges starting where the alpha is less than Threshold value. The default Threshold value is zero so starting where there is no alpha the property will expand the pixels. See the example below:
Step 1: Prepare the Texture
Load it into Photoshop or other Image Editor. Delete all non island pixels. Save the map as PNG or other format that supports alpha.
Step 2: Load the Texture as Color Source
Create a project with the map as a source color texture. Ensure that the “Alpha Threshold” property is set to zero.
Step 3: Modify the Displacement Map
Disable the “Preserve Alpha” property on the displacement map. This will show the edge expansion along the island borders. Make any changes to the displacement map as needed.
Step 4: Apply an Alpha Mask to the Normal Map
Create a Black and White mask image where all island pixels are white and all others are black. Load this into the Normal Map’s “Alpha Mask” property. This will remove all expanded pixels from the finalized normal map.