Detail normal maps are used in the same way as detail textures, to add high frequency information/detail to an otherwise low frequency base map or texture. This is useful in situations where a large range of view distances are allowed, such as a landscape, where the view point can encompass the ground close up and distant terrain.
Ideally, the base texture would be of high enough resolution to provide enough detail close to the viewer (using mipmapping to handle the lower sampling frequency at further distances). In fact, the use of mega-textures achieves just this, by texture streaming the relevant resolution segments into memory. However, this solution is not yet in wide use, next-gen engines such as id Tech 5 being among the first to implement it in real-time on current hardware.
Excluding this, the usual method is to stretch one texture over a large mesh to provide a low frequency basic colour and modulating it with a repeating texture to give the illusion of higher detail. The same is done with detail normal maps, but because the maps represent unit length direction vectors, the base and detail normals can’t simply be multiplied together the same way albedos (colours) can.
Primary normal map
Instead, the primary map perturbs (rotates) the interpolated vertex normal, passing the derived normal as input to the next normal map. Pertubation is performed by constructing a matrix composed of the current normal/binormal/tangent vectors that defines the ”basis” of the polygon. This is simply the vectors in world space representing the 3 axes that are perpendicular to the surface (normal), parallel to the surface (binormal) and a third vector also parallel to the surface but perpendicular to the other two vectors (tangent).
By multiplying the normal sampled from the normal map by this NBT matrix (actually the BTN matrix, as the normal is z-axis), the sampled normal is transformed into world space. Breaking down the matrix multiply, the transformation of the sampled normal (n1) from tangent-space into world space (Normal′) becomes straight forward:
 Normal′ = n1.x * Binormal + n1.y * Tangent + n1.z * Normal
In the case of a flat normal with no deviation, n1=(0,0,1), resulting in Normal′=Normal. The x and y components cause according Binormal & Tangent contributions, and as long as the basis axes and the sampled normal are all unit vectors, the resultant linear combination is also of unit length.
Detail normal map
After the primary normal map has been applies, you’re left with Normal′, Binormal and Tangent vectors as the new basis. As only the normal is required for the lighting equation, the fact that the Binormal and Tangent are no longer orthonormal (perpendicular) to Normal′ isn’t a problem with only a single normal map. With a second perturbation, the Binormal and Tangent need to be made orthonormal to the first perturbed normal Normal′ before the . This can be achieved by removing the Normal′ component in the Binormal vector and renormalising it. After that, it’s a simple matter of taking the cross product of the normal and binormal to derive the perturbed tangent:
 Binormal′ = normalise( Binormal - (Binormal . Normal′) * Normal′ )  Tangent′ = Normal′ x Binormal′
Strictly this may not be the geodesic rotation (the least rotation required to map the basis to align with the perturbed normal) like you can get through the use of quaternions, but it provides consistent results for a relatively small performance impact.
Surfaces in real life gather incident light from all directions, rather than a small number of directional and positional lights used in standard illumination. As a result a constant ambient term is often inappropriate, as the contributing local environment can fluctuate wildly. Ambient Occlusion (AO) [33] reduces the stark CG look without the use of full Global Illumination by generating a “smart†ambient term that varies over the model, dependant on the range of the external environment it can see. Importantly, this can be run as a pre-process, baking it into a texture that can be applied on a model at rendering time as a fast approximation to the diffuse shading term.
The preprocessing stage computes two values for each point on the model: the unoccluded fraction of hemisphere above the point and the average direction of unoccluded incident light. These terms are known as the accessibility, and its bent normal. Figure 27 illustrates this idea in 2D: an approximation to the average colour of incoming light at P can be made by averaging the light from the cone of unoccluded directions around the bent normal B.
Figure 27: Computing the accessibility and bent normal for a point (left). In some cases, the derived bent normal may point in an occluded direction (right). Images courtesy of NVIDIA.
Figure 28 illustrates a comparison of a complex model on a plane, lit by a simple diffuse shading model and AO accessibility information respectively. Notice on the diffuse shaded model the lack of weight and general unrealistic appearance, exemplified by the excessive brightness of the creature’s underside and legs.
The major benefit of this method is its speed; the precomputation of values eliminates overhead at run-time. Similarly, its low requirements mean it can run at peak performance over a wide range of graphics cards.
Figure 28: Scene shaded by diffuse model and accessibility information respectively. Note that AO is a local solution, not taking into account the occlusion effects of scene geometry.
Ambient occlusion can be extended, utilising the accessibility information to compute an approximation of the model’s appearance within a complex real-world environment, known as environment lighting. Although the accessibility factor provides the intensity of the ambient term, it is not modulated by the environment. The bent normal can be used to do a blurred lookup into an environment map to determine the incident ambient light from that direction. Figure 29 demonstrates the effect of environment lighting on a model; it provides an efficient rough approximation to the actual model’s appearance in the surroundings depicted in the environment map. The average direction may point in a direction that is actually occluded, as demonstrated in Figure 27, but otherwise the results yielded are usually realistic.
Figure 29: AO environment lighting using an environment map.
As global illumination effects are still relatively hard, dynamic ambient occlusion [34] can be used to compute AO in real-time on the GPU by treating a polygonal mesh as a series of disk-shaped elements and calculating the occlusion effect between them. By creating a hierarchical system, approximating multiple elements with single, larger disks, LOD techniques can be used to maintain frame rates with a large number of objects.
A slight modification of this method can introduce an approximation of indirect lighting by factoring in a single bounce, adding a disk-to-disk radiance transfer function. However, this then requires three passes to render: one pass to transfer the reflected or emitted light and two passes to shadow the light.
Figure 30: Dynamic AO allows animated meshes to also benefit (middle). Indirect lighting adds an extra level of realism (right).
Ambient occlusion provides a simple and elegant method of increasing the complexity of the ambient term, with low overhead. Static processing produces AO maps that fit well with texture mapping trends (specular; diffuse; normal and height ) that the field seems to be following. It also offers the ability to “drop-in†a level of global illumination without the need to redesign the renderer from the ground up.
Dynamic AO is possibly the only technique at present to provide global illumination for dynamic models with real-time frame rates. For this reason it is likely that dynamic ambient occlusion will be a major component of next-generation graphics.
[33] M. Pharr and S. Green, Ambient Occlusion, GPU Gems, Addison-Wesley Professional, 2004, pp. 279-92.
[34] M. Bunnell, Dynamic Ambient Occlusion and Indirect Lighting, GPU Gems 2, Addison-Wesley Professional, 2005, pp. 223-33.
Standard illumination techniques using direct lighting avoid the diffuse inter-reflection component due to its reliance on solving partial differentials. As a result, scenes appear darker than in real life. This can be alleviated with a constant ambient term, ensuring that no surface is completely black. However, this makes areas seem flat and does not account for contact shadows and cracks. One option to combat this is to replace the ambient light with multiple diffuse lights, but the rendering time increases disproportionately for the realism achieved, making it unsuitable for real-time usage. The focus of this section is on exploring alternate techniques that simulate a more accurate ambient term, accounting for illumination with complex light distributions in a real-time context.