Unit 8, lesson 61

Quasi Monte Carlo and effective dimension

How quasi-Monte Carlo integrates with low-discrepancy points measured by star discrepancy, how randomized quasi-Monte Carlo gives an honest standard error, and how the Brownian bridge construction changes effective dimension.

  • Video 61
  • Variance reduction and quasi-Monte Carlo

An Asian option monitored on 6464 dates becomes, once its normal inputs are written in uniform coordinates, an integral over a 6464-dimensional unit cube. Independent sampling gives an error of order N−1/2N^{-1/2} in any dimension, and evenly spread points can do better for suitable integrands, but a deterministic point set has no variance from which to read an error bar. This lesson defines the discrepancy that measures even coverage, states the Koksma-Hlawka bound with its hypothesis, derives an unbiased replicate standard error for randomized point sets, and shows how the Brownian bridge construction moves the important part of a path into the first coordinates.

Key terms

Quasi-Monte Carlo

Definition Write a model expectation as an integral over the unit cube,

I=∫[0,1]df(u) du,I = \int_{[0,1]^d} f(u)\, du,

where ff composes the discounted payoff with a map from uniform coordinates to the model inputs, for example uk↦Φ−1(uk)u_k \mapsto \Phi^{-1}(u_k) in each coordinate to get dd independent standard normals, followed by a path construction. For a point set PN={u1,…,uN}P_N = \{u_1, \dots, u_N\} in [0,1)d[0,1)^d, the equal-weight rule is

QN=1N∑j=1Nf(uj).Q_N = \frac{1}{N} \sum_{j=1}^{N} f(u_j).

Plain Monte Carlo uses independent uniform points. Quasi-Monte Carlo uses deterministic points chosen to fill the cube evenly, so its error QN−IQ_N - I is a fixed number, not a random variable. The Koksma-Hlawka inequality, stated in the course without proof, says that if ff has finite variation VHK(f)V_{HK}(f) in the sense of Hardy and Krause, then for every point set

∣QN−I∣≤VHK(f) DN∗(PN),\lvert Q_N - I \rvert \le V_{HK}(f)\, D_N^*(P_N),

where DN∗D_N^* is the star discrepancy of the next card. In one dimension VHK(f)V_{HK}(f) is the ordinary total variation. The hypothesis is restrictive for financial integrands. An unbounded integrand has infinite variation, and a payoff composed with Φ−1\Phi^{-1} is often unbounded near the faces of the cube. An indicator of a region whose boundary is not parallel to the coordinate axes, such as a digital payoff on a basket of two assets, also has infinite Hardy-Krause variation in two or more dimensions. Then the bound says nothing, which is not the same as saying that quasi-Monte Carlo fails.

In plain words Instead of trusting chance to cover the cube, quasi-Monte Carlo places the points so that they cover it evenly. The Koksma-Hlawka bound turns even coverage into an error guarantee, but only for integrands whose variation is under control, and many option payoffs do not qualify.

Example In one dimension take f(u)=u2f(u) = u^2, whose integral is 13\tfrac13, and the NN midpoints uj=(2j−1)/(2N)u_j = (2j - 1)/(2N). Averaging their squares gives

QN=13−112N2,Q_N = \frac13 - \frac{1}{12N^2},

so Q4=0.328125Q_4 = 0.328125 with error −1/192-1/192 and Q8=0.33203125Q_8 = 0.33203125 with error −1/768-1/768. The function u2u^2 increases from 00 to 11, so its total variation is 11, and the midpoints have star discrepancy 1/(2N)1/(2N). For N=4N = 4 the bound is 18=0.125\tfrac18 = 0.125, which is true but far above the actual error of about 0.00520.0052. This is an exactly computable grid example in one dimension, and the smooth integrand is integrated much better than the bound guarantees.

Star discrepancy

Definition For a point set PN={u1,…,uN}P_N = \{u_1, \dots, u_N\} in [0,1)d[0,1)^d and a corner t∈[0,1]dt \in [0,1]^d, let A(t)A(t) be the number of points in the anchored box [0,t1)×⋯×[0,td)[0, t_1) \times \dots \times [0, t_d), whose volume is t1t2⋯tdt_1 t_2 \cdots t_d. With the local mismatch

Δ(t)=A(t)N−t1t2⋯td,\Delta(t) = \frac{A(t)}{N} - t_1 t_2 \cdots t_d ,

the star discrepancy is

DN∗=sup⁡t∈[0,1]d∣Δ(t)∣.D_N^* = \sup_{t \in [0,1]^d} \lvert \Delta(t) \rvert .

It is a number in [0,1][0, 1] that depends only on where the points are, not on how they were produced, so it measures coverage and says nothing about independence. In one dimension, with the points sorted as x(1)≤⋯≤x(N)x_{(1)} \le \dots \le x_{(N)},

DN∗=12N+max⁡i∣x(i)−2i−12N∣,\begin{aligned} D_N^* &= \frac{1}{2N} \\[0.4em] &\quad + \max_{i} \left\lvert x_{(i)} - \frac{2i - 1}{2N} \right\rvert , \end{aligned}

so 1/(2N)1/(2N) is the smallest possible value, attained only by the midpoints. In one dimension it is the Kolmogorov-Smirnov distance between the empirical distribution of the points and the uniform distribution.

In plain words Look at every box with a corner at the origin and compare the fraction of points inside it with its volume. The star discrepancy is the worst mismatch, so a small value means that no corner box holds noticeably too many or too few points.

Example The four midpoints 0.1250.125, 0.3750.375, 0.6250.625 and 0.8750.875 have D4∗=18D_4^* = \tfrac18, the minimum. The four left endpoints 00, 0.250.25, 0.50.5 and 0.750.75 have D4∗=14D_4^* = \tfrac14, because the interval [0,t)[0, t) with tt just above 0.750.75 holds all four points but has length only slightly more than 0.750.75. Even coverage in each coordinate separately is not enough. In two dimensions place an even number NN of points on the diagonal at ((2j−1)/(2N),(2j−1)/(2N))\bigl((2j - 1)/(2N), (2j - 1)/(2N)\bigr). Each coordinate on its own is the ideal midpoint set, but the box [0,12)2[0, \tfrac12)^2 holds half of the points and has area 14\tfrac14, so DN∗≥14D_N^* \ge \tfrac14 for every NN, and the set never covers the square evenly.

Low-discrepancy points

Definition An infinite sequence u1,u2,…u_1, u_2, \dots in [0,1)d[0,1)^d is a low-discrepancy sequence if there is a constant CdC_d such that the first NN points satisfy

DN∗≤Cd (log⁡N)dND_N^* \le C_d\, \frac{(\log N)^d}{N}

for all N≥2N \ge 2. The van der Corput sequence in base 22 mirrors the binary digits of jj about the binary point, so j=1,2,3,4j = 1, 2, 3, 4, written 11, 1010, 1111 and 100100 in binary, give 0.10.1, 0.010.01, 0.110.11 and 0.0010.001 in binary, that is 12\tfrac12, 14\tfrac14, 34\tfrac34 and 18\tfrac18. The Halton sequence uses van der Corput sequences in different prime bases for different coordinates, and Sobol’ sequences are built from base 22 digit operations. Both satisfy the bound. For independent uniform points the star discrepancy is typically of order N−1/2N^{-1/2}. Combined with the Koksma-Hlawka inequality, an integrand of finite Hardy-Krause variation is integrated with error at most a constant times (log⁡N)d/N(\log N)^d / N. That beats N−1/2N^{-1/2} as N→∞N \to \infty, but for large dd the factor (log⁡N)d(\log N)^d is enormous at practical sizes, so this rate alone does not predict performance in high dimension, and general convergence rates lie beyond this lesson.

In plain words A low-discrepancy sequence keeps filling the gaps it has left, so every initial stretch of it covers the cube evenly. Its coverage error shrinks almost like 1/N1/N instead of like 1/N1/\sqrt{N}, up to logarithmic factors that matter in high dimension.

Example The first seven van der Corput points are 12\tfrac12, 14\tfrac14, 34\tfrac34, 18\tfrac18, 58\tfrac58, 38\tfrac38 and 78\tfrac78, exactly the multiples k/8k/8 for k=1,…,7k = 1, \dots, 7, and their star discrepancy is D7∗=18D_7^* = \tfrac18. Each new point bisects one of the largest remaining gaps. The first 1515 points are the multiples of 116\tfrac{1}{16} other than 00, with D15∗=116=0.0625D_{15}^* = \tfrac{1}{16} = 0.0625, while 1515 independent uniform points have an expected star discrepancy of about 0.210.21.

Randomized quasi-Monte Carlo

Definition A randomization of a point set PNP_N produces random points u~1,…,u~N\tilde u_1, \dots, \tilde u_N such that each u~j\tilde u_j is uniformly distributed on [0,1)d[0,1)^d while the set keeps an even structure. The simplest is the random shift modulo one, u~j=(uj+Δ) mod 1\tilde u_j = (u_j + \Delta) \bmod 1 in each coordinate with Δ\Delta uniform on [0,1)d[0,1)^d, which leaves the spacing of a grid or lattice unchanged. Scrambling of digital nets is another, whose implementation lies beyond this lesson. For an integrable ff each term has expectation II, so

QN=1N∑j=1Nf(u~j)Q_N = \frac{1}{N} \sum_{j=1}^{N} f(\tilde u_j)

is unbiased. The points of one randomized set are dependent, so the NN values f(u~j)f(\tilde u_j) cannot be treated as independent draws. Instead take R≥2R \ge 2 independent randomizations of the same point set, with the same NN and the same integrand, and let QN(r)Q_N^{(r)} be the estimate from the rr-th one. These replicates are independent and identically distributed with mean II. If their variance is finite, the average

QˉR=1R∑r=1RQN(r)\bar Q_R = \frac{1}{R} \sum_{r=1}^{R} Q_N^{(r)}

has variance Var⁡(QN(1))/R\operatorname{Var}(Q_N^{(1)})/R, and this variance is estimated without bias by

SE⁡^2=1R(R−1)×∑r=1R(QN(r)−QˉR)2,\begin{aligned} \widehat{\operatorname{SE}}^2 &= \frac{1}{R(R-1)} \\[0.4em] &\quad \times \sum_{r=1}^{R} \bigl(Q_N^{(r)} - \bar Q_R\bigr)^2 , \end{aligned}

because the sample variance with divisor R−1R - 1 is unbiased for the variance of independent, identically distributed replicates. A Student tt interval with R−1R - 1 degrees of freedom is exact only for normal replicates and is otherwise an approximation that needs enough replicates. Randomization makes the error measurable while keeping the estimate unbiased, but it does not guarantee one convergence rate for every payoff.

In plain words One randomized point set gives one unbiased estimate, but its points are tied together and cannot judge their own error. Repeating the randomization independently gives a few independent estimates, and their spread is the error bar.

Example Shift the grid 00, 0.250.25, 0.50.5, 0.750.75 by a uniform Δ\Delta modulo one and integrate u2u^2. The shifted set is s+k/4s + k/4 for k=0,1,2,3k = 0, 1, 2, 3, with s=Δ mod 14s = \Delta \bmod \tfrac14 uniform on [0,14)[0, \tfrac14), so

Q4=s2+0.75 s+0.21875.Q_4 = s^2 + 0.75\, s + 0.21875 .

Its mean is 148+0.758+0.21875=13\tfrac{1}{48} + \tfrac{0.75}{8} + 0.21875 = \tfrac13, the exact integral, and its variance is 241/46080≈0.00523241/46080 \approx 0.00523, about 4.254.25 times smaller than the variance 1/45≈0.02221/45 \approx 0.0222 of an average of four independent uniform points. For the arithmetic of the error bar, take the four constructed replicate estimates 0.3320.332, 0.3340.334, 0.3310.331 and 0.3330.333, which are not the output of an actual randomization. Their mean is 0.33250.3325, their sample variance is about 1.6667×10−61.6667 \times 10^{-6}, and

SE⁡^=1.6667×10−64≈0.000645.\begin{aligned} \widehat{\operatorname{SE}} &= \sqrt{\frac{1.6667 \times 10^{-6}}{4}} \\[0.4em] &\approx 0.000645 . \end{aligned}

Brownian bridge construction

Definition Let WW be a standard Brownian motion, 0=t0<t1<⋯<tm=T0 = t_0 < t_1 < \dots < t_m = T a time grid and Z1,…,ZmZ_1, \dots, Z_m independent standard normals. The chronological construction sets Wtk=Wtk−1+tk−tk−1 ZkW_{t_k} = W_{t_{k-1}} + \sqrt{t_k - t_{k-1}}\, Z_k. The Brownian bridge construction generates WT=T Z1W_T = \sqrt{T}\, Z_1 first and then fills in interior times by Gaussian conditioning. Suppose s<u<ts < u < t are grid times, the values Ws=xW_s = x and Wt=yW_t = y are already generated, and every other generated value lies at a time outside (s,t)(s, t). By independent increments, the conditional law of WuW_u given all generated values is N(μu,vu)N(\mu_u, v_u) with

μu=(t−u) x+(u−s) yt−s,vu=(u−s)(t−u)t−s,\begin{aligned} \mu_u &= \frac{(t - u)\, x + (u - s)\, y}{t - s}, \\[0.4em] v_u &= \frac{(u - s)(t - u)}{t - s}, \end{aligned}

and the next coordinate fills it in as Wu=μu+vu ZkW_u = \mu_u + \sqrt{v_u}\, Z_k. For the midpoint of [0,T][0, T] with W0=0W_0 = 0 and WT=wW_T = w, Gaussian conditioning with Cov⁡(WT/2,WT)=T/2\operatorname{Cov}(W_{T/2}, W_T) = T/2 and Var⁡(WT)=T\operatorname{Var}(W_T) = T gives the mean T/2T w=w/2\tfrac{T/2}{T}\, w = w/2 and the variance T/2−(T/2)2/T=T/4T/2 - (T/2)^2/T = T/4. On a grid with m=2km = 2^k steps the fill-in goes level by level, the midpoint first, then the quarter points, and so on. Both constructions produce a centered Gaussian vector with covariance min⁡(ti,tj)\min(t_i, t_j), so they give exactly the same joint law of (Wt1,…,Wtm)(W_{t_1}, \dots, W_{t_m}), and the same expectation and variance for every square-integrable payoff. They differ only in which coordinate controls which feature of the path.

In plain words First decide where the path ends, then where it is halfway, then at the quarter points, each time drawing only the deviation from the straight line between values already known. The first coordinates fix the broad shape of the path and the later ones add finer wiggles, while the law of the whole path stays the same.

Example With T=1T = 1 and W1=0.6W_1 = 0.6, the midpoint has the conditional law N(0.3,0.25)N(0.3, 0.25), so W1/2=0.3+0.5 Z2W_{1/2} = 0.3 + 0.5\, Z_2. With two steps the bridge gives W1=Z1W_1 = Z_1 and W1/2=12Z1+12Z2W_{1/2} = \tfrac12 Z_1 + \tfrac12 Z_2, so Var⁡(W1/2)=14+14=12\operatorname{Var}(W_{1/2}) = \tfrac14 + \tfrac14 = \tfrac12 and Cov⁡(W1/2,W1)=12\operatorname{Cov}(W_{1/2}, W_1) = \tfrac12, exactly as for Brownian motion. Once the midpoint value W1/2=w′W_{1/2} = w' is known, the quarter point has the conditional law N(w′/2,0.125)N(w'/2, 0.125), since (0.25⋅0.25)/0.5=0.125(0.25 \cdot 0.25)/0.5 = 0.125.

Effective dimension

Definition Let ff be square integrable on [0,1]d[0,1]^d, with variance σ2=Var⁡(f(U))>0\sigma^2 = \operatorname{Var}(f(U)) > 0 for UU uniform on the cube. The ANOVA decomposition writes f=∑SfSf = \sum_{S} f_S over the subsets S⊆{1,…,d}S \subseteq \{1, \dots, d\}, where f∅=If_{\varnothing} = I and, for S≠∅S \neq \varnothing, the term fSf_S depends only on the coordinates in SS and integrates to zero over each of them. These terms are orthogonal, so

σ2=∑S≠∅σS2,σS2=Var⁡(fS(U)).\begin{gathered} \sigma^2 = \sum_{S \neq \varnothing} \sigma_S^2, \\[0.4em] \sigma_S^2 = \operatorname{Var}\bigl(f_S(U)\bigr). \end{gathered}

For a proportion pp close to 11, such as 0.990.99, the effective dimension in the truncation sense is the smallest ss for which the terms with ∅≠S⊆{1,…,s}\varnothing \neq S \subseteq \{1, \dots, s\} carry at least pσ2p\sigma^2. In the superposition sense it is the smallest ss for which the terms with 1≤∣S∣≤s1 \le \lvert S \rvert \le s carry at least pσ2p\sigma^2. The truncation dimension depends on the order of the coordinates, so it changes when the path is built differently, while the integral does not.

In plain words Effective dimension counts how many leading coordinates, or how large groups of interacting coordinates, are needed to explain nearly all of the payoff’s variance. A 6464-dimensional integral whose variance sits in its first few coordinates behaves much like a low-dimensional one, and the leading coordinates and low-dimensional projections of common low-discrepancy constructions are usually their best distributed parts.

Example Take T=1T = 1, m=16m = 16 equal steps and the path average Aˉ=116∑kWtk\bar A = \tfrac{1}{16} \sum_{k} W_{t_k}, which up to the factor σ\sigma and a constant is the log of a geometric average price. It is a linear combination ∑kckZk\sum_k c_k Z_k of the normal coordinates, so its ANOVA terms are one-dimensional with σ{k}2=ck2\sigma_{\{k\}}^2 = c_k^2, and its variance is 187/512≈0.3652187/512 \approx 0.3652 in both constructions. In the chronological construction the first coordinate explains about 17.117.1 percent of that variance, and 1313 coordinates are needed for 9999 percent. In the bridge construction the first coordinate, which sets W1W_1, explains about 77.377.3 percent, and 66 coordinates reach 9999 percent. For the terminal value W1W_1 the contrast is 1616 coordinates against 11. The bridge is not better for everything. The single increment W9/16−W8/16W_{9/16} - W_{8/16} uses one coordinate in the chronological construction and five in the bridge construction.

Check yourself

  1. Five independent random shifts of the same point set give the price estimates 1.2041.204, 1.1981.198, 1.2011.201, 1.2071.207 and 1.1951.195 (constructed numbers), each from 10241024 points. Find the combined estimate and its standard error. Why is the sample standard deviation of all 51205120 payoff evaluations divided by 5120\sqrt{5120} not a valid standard error?
Show the answer

The mean is Qˉ5=1.201\bar Q_5 = 1.201. The deviations are 0.0030.003, −0.003-0.003, 00, 0.0060.006 and −0.006-0.006, whose squares add to 9×10−59 \times 10^{-5}. Then

SE⁡^2=9×10−55⋅4=4.5×10−6,SE⁡^≈0.00212.\begin{aligned} \widehat{\operatorname{SE}}^2 &= \frac{9 \times 10^{-5}}{5 \cdot 4} \\[0.4em] &= 4.5 \times 10^{-6}, \\[0.4em] \widehat{\operatorname{SE}} &\approx 0.00212 . \end{aligned}

If the replicates are roughly normal, a Student tt interval with 44 degrees of freedom, whose 0.9750.975 quantile is about 2.7762.776, gives approximately 1.201±0.00591.201 \pm 0.0059. The pooled formula assumes that all 51205120 evaluations are independent, but the 10241024 points of each randomized set are dependent by construction, because they are placed to avoid each other. The pooled formula estimates the error of plain Monte Carlo with independent points, not the error of this estimator, so only the five independent replicates measure it.

  1. Brownian motion is built with the bridge construction on [0,2][0, 2], and the first coordinate gives W2=−0.8W_2 = -0.8. Find the conditional law of W1W_1. If the fill-in gives W1=−0.5W_1 = -0.5, find the conditional laws of W0.5W_{0.5} and W1.5W_{1.5}. A colleague claims that the bridge also lowers the variance of a plain Monte Carlo estimate with independent draws. Is that right?
Show the answer

The midpoint of [0,2][0, 2] with W0=0W_0 = 0 and W2=−0.8W_2 = -0.8 has mean −0.8/2=−0.4-0.8/2 = -0.4 and variance T/4=0.5T/4 = 0.5, so W1∼N(−0.4,0.5)W_1 \sim N(-0.4, 0.5). Given W0=0W_0 = 0 and W1=−0.5W_1 = -0.5, the point 0.50.5 has mean −0.25-0.25 and variance (0.5⋅0.5)/1=0.25(0.5 \cdot 0.5)/1 = 0.25. Given W1=−0.5W_1 = -0.5 and W2=−0.8W_2 = -0.8, the point 1.51.5 has mean 12(−0.5−0.8)=−0.65\tfrac12(-0.5 - 0.8) = -0.65 and the same variance 0.250.25. The colleague is wrong. Both constructions give the same joint law of the path, so a payoff has the same distribution either way, and with independent draws its sample mean has the same variance. The bridge only changes which coordinate controls which part of the path, which matters for point sets whose coordinates are not equally well spread.

  1. Take T=1T = 1 with the two dates 12\tfrac12 and 11, and normal coordinates Z1,Z2Z_1, Z_2. For the sum W1/2+W1W_{1/2} + W_1 and for W1/2W_{1/2} alone, write each as a combination of Z1Z_1 and Z2Z_2 in the chronological and in the bridge construction, and find the share of the variance carried by the first coordinate.
Show the answer

Chronologically W1/2=0.5 Z1W_{1/2} = \sqrt{0.5}\, Z_1 and W1=0.5 (Z1+Z2)W_1 = \sqrt{0.5}\,(Z_1 + Z_2). The bridge gives W1=Z1W_1 = Z_1 and W1/2=12Z1+12Z2W_{1/2} = \tfrac12 Z_1 + \tfrac12 Z_2. For the sum, the chronological form is 2 Z1+0.5 Z2\sqrt{2}\, Z_1 + \sqrt{0.5}\, Z_2 and the bridge form is 1.5 Z1+0.5 Z21.5\, Z_1 + 0.5\, Z_2. Both have variance 2.52.5, as they must, but the first coordinate carries 2/2.5=0.82/2.5 = 0.8 of it chronologically and 2.25/2.5=0.92.25/2.5 = 0.9 with the bridge. For W1/2W_{1/2} alone, the chronological form 0.5 Z1\sqrt{0.5}\, Z_1 puts all of the variance 0.50.5 on the first coordinate, while the bridge form 12Z1+12Z2\tfrac12 Z_1 + \tfrac12 Z_2 splits it equally, a share of 0.50.5. The rearrangement helps the path sum and hurts the value at the early date, so the choice of construction depends on the payoff.

How this lesson connects