Image-Based Lighting

Last Update:May 18, 2024 am

Image Based Real-time Environment Lighting

Image-based lighting refers to storing the radiance of environment light from each direction with a texture, which is then used for shading calculation. This texture is called environment texture.

The original reflection equation is

Here we do not consider lighting other than environment light. And we assume that the environment light is the same for any shading point $\mathbf{p}$. Thus the environment texture can be denoted as $L_i(\boldsymbol{\omega}_i)$.Accordingly, the reflection equation can be written as

In this article, the geometric information at the point $\mathbf{p}$ that we need is only the normal vector $\mathbf{n}$. So the above formula can be rewritten as

Diffuse Reflection

Simplified Fresnel Term

If we only consider the diffuse reflection, the reflection equation is

where

In real-time rendering, a common approach is to precompute complex integrals. Generally, for any $\mathbf{n}$ and , we aim to obtain the corresponding $L_o(\mathbf{n}, \boldsymbol{\omega}_o)$. On one hand, there are infinitely many possible values for $\boldsymbol{\omega}_o$. On the other hand, considering the rotation of objects, $\mathbf{n}$ also has infinitely many values. It is clearly impossible to precompute for all $\mathbf{n}$ and $\boldsymbol{\omega}_o$. The most naive idea is to precompute for a limited number of $\mathbf{n}$ and $\boldsymbol{\omega}_o$ and then interpolate the results. However, this requires creating a 4D LUT (lookup table), which consumes a vast amount of space and time. A more economical method is to simplify the integrand in the reflection equation:

allowing the terms involving $\boldsymbol{\omega}_o$ to be moved outside the integral. Thus, we only need to precompute complex integrals for a limited number of $\mathbf{n}$ and store the results in a 2D LUT. The computations regarding $\boldsymbol{\omega}_o$ can be performed in real time.

Note that in the integrand, only the Fresnel term includes $\boldsymbol{\omega}_o$. Since $\mathbf{h}$ contains $\boldsymbol{\omega}_i$, this term cannot be directly moved outside the integral. We can replace the original Fresnel term with a simplified version , which is expressed as follows:

Thus, the reflection equation simplifies to:

Afterwards, we only need to precompute the following integral for a finite number of $\mathbf{n}$:

Note that this integral represents the irradiance received by a shading point with normal $\mathbf{n}$, hence it is also referred to as an irradiance map.

If the environmental texture is a cubic box map, each of its texels corresponds to a direction vector. We can let $\mathbf{n}$ cover all these direction vectors and store the value of $I_{\mathrm{diff}}\left(\mathbf{n}\right)$ in the corresponding texel. During rendering, $L_o(\mathbf{n},\boldsymbol{\omega}_o)$ is calculated as follows:

Precomputation of the Irradiance Map

We can compute using the Monte Carlo integration method. Given $\mathbf{n}$, let $\eta_{\mathbf{n}}$ be a probability distribution over the hemisphere $H^2$ with the probability density function as

We utilize for importance sampling to obtain the statistic of $I_{\mathrm{diff}}\left(\mathbf{n}\right)$:

where $N$ is the number of samples. Below, we will specifically describe how to generate samples for . We will provide two methods: the coordinate transformation method and the quaternion rotation method.

Coordinate Transformation Method
Define

The matrix $T_{\mathbf{n}}$ of the basis vectors of the tangent space in world space can be represented as

where

Suppose are independent and uniformly distributed $U(0,1)$, then

follows the distribution $\eta_{\mathbf{n}}$.

Quaternion Rotation Method

  • When $\mathbf{n}=(0,0,1)$, we can use inverse transform sampling. Suppose are independent and uniformly distributed $U(0,1)$, then

    follows the distribution $\eta_{(0,0,1)}$.

  • When $\mathbf{n}=(0,0,-1)$, follows the distribution $\eta_{(0,0,-1)}$.

  • When $\mathbf{n}=(n_x,n_y,n_z)\ne(0,0,\pm 1)$, rotate the vector $(0,0,1)$ along the rotation axis

    to vector $\mathbf{n}$, the quaternion is

    where $\alpha$ is the rotation angle. Since

    we have

    From this, we can compute the rotation matrix for the quaternion rotation $\mathrm{rot}_q:p\mapsto qpq^{-1}$

    It is evident that follows the distribution $\eta_{\mathbf{n}}$.

Specular Reflection

Now we consider only specular reflection. The reflection equation is given by

To simplify the equation, we can bring $L_i(\boldsymbol{\omega}_i)$ out of the integral. Thus, we express $L_o(\mathbf{n},\boldsymbol{\omega}_o)$ as follows

We aim to compute the parts containing $L_i(\boldsymbol{\omega}_i)$ and those not containing $L_i(\boldsymbol{\omega}_i)$ separately.

Split Sum - Part 1

Formula Derivation

Direct calculation

is relatively complex. To reduce the dimensionality of the precomputed parameter space, the assumption here is that the lobe shape remains unchanged, always being the lobe when the ray is incident along the normal, i.e., when calculating the integral, let the values of $\boldsymbol{\omega}_o, \mathbf{n}$ be equal to the value of the reflection vector $\mathbf{r}(\boldsymbol{\omega}_o)=2(\boldsymbol{\omega}_o\cdot\mathbf{n})\mathbf{n}-\boldsymbol{\omega}_o$ of $\boldsymbol{\omega}_o$. In other words, use $\mathbf{r}(\boldsymbol{\omega}_o)$ to replace $\boldsymbol{\omega}_o$ and $\mathbf{n}$ in the integrand. Let $\mathbf{n}^*=\mathbf{r}(\boldsymbol{\omega}_o)$,

where

$C(\boldsymbol{\omega}_i,\mathbf{n}^*)$ is defined as

Since

where $\cos\theta_{\mathbf{h}}=\mathbf{n}^* \cdot \mathbf{h}$, we can use Monte Carlo integration to importance sample

Noting that

we have

where

while $\mathbf{h}^{(k)}$ is a random variable on the hemisphere $H^2_{\mathbf{n}^*}$, following the probability distribution with probability density function

Sampling Scheme

To apply the inverse transform sampling method, first parameterize $\mathbf{h}^{(k)}$ in the tangent space at $\mathbf{n}^*$ with spherical coordinates . Let be independent and follow the uniform distribution $U(0,1)$. From

we know are independent. So we can derive the marginal distributions of respectively

Using the inverse transform of the cdf, we can obtain the sampling scheme of $\mathbf{h}^{(k)}$ in spherical coordinates

From this, we can derive the sampling scheme of $\boldsymbol{\omega}_i^{(k)}$ in spherical coordinates

Since the above spherical coordinates are generated in the tangent space, we need to first convert them to Cartesian coordinates, and then use coordinate transformation to convert them to world space coordinates. Using the previously defined $T_{\mathbf{n}}$, the sampling scheme of $\boldsymbol{\omega}_i^{(k)}$ in world space can be written as

Finally, we can derive the estimate for the first part of the split integral

where

We can precompute for some $\alpha$, generating multiple cubemap mip levels like a mipmap. For other $\alpha$, the value of $S(\mathbf{n}^*,\alpha)$ can be obtained by trilinear interpolation. During rendering, the first part of the split integral can be estimated as

Split Sum - Part 2

Formula Derivation

The right-hand side of the split integral can be simplified to

Importance sampling is given by


Copyright Notice: All articles in this blog are licensed under CC BY-SA 4.0 unless declaring additionally.