Multivariable Taylor Approximation, multivariable and vector calculus lesson

A tangent plane matches a surface’s height and tilt at one point, but it cannot see whether the surface curves up like a bowl or bends like a saddle. Adding second derivatives fixes that. The resulting quadratic model is the natural next step when a linear approximation is too crude, and it is the tool behind the second derivative test for extrema. The same idea continues to cubic and higher models, and Taylor’s theorem says how large the error can be.

Key terms

Symmetry of mixed partials

Definition Let $f$ be defined on an open set $U \subseteq \mathbb{R}^n$ . If the second partial derivatives $f_{x_i x_j}$ and $f_{x_j x_i}$ exist and are continuous on $U$ , then

f_{x_i x_j} = f_{x_j x_i} \quad \text{on } U.

This is Clairaut’s theorem. In particular it holds for every $f$ of class $C^2$ on $U$ , and for $f$ of class $C^k$ on $U$ every partial derivative of order at most $k$ is independent of the order of differentiation. Without continuity the mixed partials can differ.

In plain words For well behaved functions it does not matter whether you differentiate in $x$ first or in $y$ first. The continuity hypothesis is not decoration, it is what makes the result true.

Example For $f(x, y) = x^2 y^3$ both orders give $6xy^2$ . Now take $f(x, y) = \frac{xy(x^2 - y^2)}{x^2 + y^2}$ for $(x, y) \ne (0, 0)$ and $f(0, 0) = 0$ . A direct computation gives $f_x(0, y) = -y$ and $f_y(x, 0) = x$ . Differentiating the first in $y$ gives $-1$ at the origin, and differentiating the second in $x$ gives $1$ . The two mixed partials disagree, and indeed they are not continuous at $(0, 0)$ .

Hessian

Definition Let $f$ have all second partial derivatives at a point $a$ of an open set $U \subseteq \mathbb{R}^n$ . The Hessian matrix of $f$ at $a$ is the $n \times n$ matrix

Hf(a) = \Big[ f_{x_i x_j}(a) \Big]_{i, j = 1}^{n}.

For two variables,

Hf(a) = \begin{bmatrix} f_{xx}(a) & f_{xy}(a) \\ f_{yx}(a) & f_{yy}(a) \end{bmatrix}.

When $f$ is $C^2$ on $U$ , Clairaut’s theorem makes $Hf(a)$ a symmetric matrix.

In plain words The Hessian collects all the second derivatives in one table. It plays the role that $f''$ plays in one variable.

Example For $f(x, y) = x e^y$ the second partials are $f_{xx} = 0$ , $f_{xy} = f_{yx} = e^y$ and $f_{yy} = x e^y$ . At $(1, 0)$

Hf(1, 0) = \begin{bmatrix} 0 & 1 \\ 1 & 1 \end{bmatrix}.

Quadratic forms

Definition A quadratic form on $\mathbb{R}^n$ is a function $Q(h) = h^{T} A h$ , where $A$ is a symmetric $n \times n$ matrix and $h$ is a column vector. In coordinates $Q(h) = \sum_{i, j} a_{ij} h_i h_j$ . For $n = 2$ and

A = \begin{bmatrix} a & b \\ b & c \end{bmatrix}

the two off diagonal entries contribute equally, and

Q(h) = a h_1^2 + 2b h_1 h_2 + c h_2^2.

In plain words A quadratic form is a polynomial in which every term has degree exactly two. The factor $2$ in front of the mixed term comes from the two equal entries $b$ above and below the diagonal.

Example Take

A = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}.

The four entries give $2h_1^2 + h_1 h_2 + h_2 h_1 + 3h_2^2$ , which combines to $2h_1^2 + 2h_1 h_2 + 3h_2^2$ . At $h = \langle 1, -1 \rangle$ the value is $2 - 2 + 3 = 3$ .

Second-order Taylor polynomial

Definition Let $f$ be $C^2$ on an open set containing $a$ , and write $h = x - a$ . The second-order Taylor polynomial of $f$ centered at $a$ is

\begin{aligned} T_2(x) &= f(a) + \nabla f(a) \cdot h \\ &\quad + \tfrac{1}{2} \, h^{T} Hf(a) \, h. \end{aligned}

For two variables with $a = (x_0, y_0)$ , $h_1 = x - x_0$ and $h_2 = y - y_0$ , the quadratic part is

\tfrac{1}{2} \big( f_{xx} h_1^2 + 2 f_{xy} h_1 h_2 + f_{yy} h_2^2 \big),

with all derivatives taken at $a$ . It is the only polynomial of degree at most $2$ whose value and first and second partial derivatives at $a$ agree with those of $f$ .

In plain words Start with the height at the center, add the tangent plane, then add a quadratic correction that bends the model the way the surface bends. Always write the powers of $x - x_0$ and $y - y_0$ , never of $x$ and $y$ , unless the center is the origin.

Example Take $f(x, y) = x e^y$ centered at $(1, 0)$ . Here $f(1, 0) = 1$ , $\nabla f(1, 0) = \langle 1, 1 \rangle$ and the Hessian is the one above, so

\begin{aligned} T_2 &= 1 + (x - 1) + y \\ &\quad + (x - 1) y + \tfrac{1}{2} y^2. \end{aligned}

At $(1.1, 0.1)$ the model gives $1 + 0.1 + 0.1 + 0.01 + 0.005 = 1.215$ , while the true value is $1.1 e^{0.1} \approx 1.215688$ .

Multi-index notation

Definition A multi-index is an $n$ -tuple $\alpha = (\alpha_1, \dots, \alpha_n)$ of nonnegative integers. Its order is $\lvert \alpha \rvert = \alpha_1 + \dots + \alpha_n$ , and for $h$ in $\mathbb{R}^n$

\begin{gathered} \alpha! = \alpha_1! \cdots \alpha_n!, \\ h^{\alpha} = h_1^{\alpha_1} \cdots h_n^{\alpha_n}. \end{gathered}

The partial derivative $D^{\alpha} f$ differentiates $\alpha_i$ times in $x_i$ for each $i$ . If $f$ is $C^k$ on an open set containing $a$ , the Taylor polynomial of order $k$ centered at $a$ is

T_k(a + h) = \sum_{\lvert \alpha \rvert \le k} \frac{D^{\alpha} f(a)}{\alpha!} \, h^{\alpha}.

In plain words A multi-index records how many times you differentiate in each variable. Each mixed derivative appears once in the sum, and dividing by $\alpha!$ accounts for all the orders in which the same derivatives could be taken.

Example Find the cubic model of $f(x, y) = e^x \sin y$ at the origin. The nonzero coefficients come from $\alpha = (0, 1)$ , $(1, 1)$ , $(2, 1)$ and $(0, 3)$ , with derivatives $1, 1, 1, -1$ at the origin, so

T_3 = y + xy + \frac{x^2 y}{2} - \frac{y^3}{6}.

For $\alpha = (2, 1)$ we have $\alpha! = 2! \, 1! = 2$ , which gives the $\frac{1}{2}$ . The same answer comes from multiplying $1 + x + \frac{x^2}{2}$ by $y - \frac{y^3}{6}$ and keeping terms of degree at most $3$ . For $k = 2$ the formula agrees with the Hessian version, because $\alpha = (1, 1)$ has $\alpha! = 1$ and so carries the full mixed term $f_{xy} h_1 h_2$ .

Remainder

Definition Let $f$ be $C^{k + 1}$ on an open set containing the segment from $a$ to $a + h$ . Then there is a number $\theta$ with $0 < \theta < 1$ such that $f(a + h) = T_k(a + h) + R_k(h)$ , where

R_k(h) = \sum_{\lvert \alpha \rvert = k + 1} \frac{D^{\alpha} f(a + \theta h)}{\alpha!} \, h^{\alpha}.

This is Taylor’s theorem with the Lagrange form of the remainder. For $k = 1$ it reads $R_1(h) = \tfrac{1}{2} \, h^{T} Hf(a + \theta h) \, h$ .

In plain words The error looks like the next term of the series, except that the derivatives are evaluated at some unknown point on the segment instead of at the center.

Example For $f(x, y) = e^{x + y}$ at the origin, $T_1 = 1 + x + y$ . Every second partial is $e^{x + y}$ , so

R_1(h) = \tfrac{1}{2} e^{\theta (h_1 + h_2)} (h_1 + h_2)^2.

At $h = \langle 0.1, 0.1 \rangle$ the actual error is $e^{0.2} - 1.2 \approx 0.021403$ . The formula gives $0.02 \, e^{0.2 \theta}$ , which matches for $\theta \approx 0.339$ , a number strictly between $0$ and $1$ as promised.

Approximation error

Definition There are two kinds of error statements. The asymptotic one says that if $f$ is $C^k$ on an open set containing $a$ , then

\lim_{h \to 0} \frac{f(a + h) - T_k(a + h)}{\lVert h \rVert^k} = 0.

It describes how fast the error shrinks but gives no number for a particular $h$ . The quantitative one follows from the remainder. Assume $f$ is $C^{k + 1}$ on an open set containing the segment from $a$ to $a + h$ . If $\lvert D^{\alpha} f \rvert \le M$ at every point of the segment for every $\alpha$ with $\lvert \alpha \rvert = k + 1$ , then

\begin{aligned} \lvert R_k(h) \rvert &\le \frac{M}{(k + 1)!} \\[0.4em] &\quad \cdot \big( \lvert h_1 \rvert + \dots + \lvert h_n \rvert \big)^{k + 1}. \end{aligned}

In plain words The first statement says the model gets very good very fast as you approach the center. The second gives an actual guaranteed error, once you bound the next derivatives along the segment you use.

Example For $f(x, y) = x e^y$ and its quadratic model at $(1, 0)$ , the third partials are $0$ , $0$ , $e^y$ and $x e^y$ . On the segment to $(1.1, 0.1)$ we have $x \le 1.1$ and $y \le 0.1$ , so all of them are at most $M = 1.22$ . With $h = \langle 0.1, 0.1 \rangle$ ,

\lvert R_2 \rvert \le \frac{1.22}{6} (0.2)^3 \approx 0.00163.

The actual error found earlier is about $0.000688$ , safely inside the bound.

Check yourself

Find the second-order Taylor polynomial of $f(x, y) = \ln(1 + x + 2y)$ centered at the origin.

Show the answer

At the origin $f = 0$ and $\nabla f = \langle 1, 2 \rangle$ . Every second partial is $-\frac{1}{(1 + x + 2y)^2}$ times $1$ , $2$ or $4$ , so the Hessian at the origin is

\begin{bmatrix} -1 & -2 \\ -2 & -4 \end{bmatrix}.

Therefore

T_2 = x + 2y - \tfrac{1}{2} (x + 2y)^2.

Write $h^{T} A h$ as a polynomial and evaluate it at $h = \langle 2, 1 \rangle$ , where

A = \begin{bmatrix} 1 & 2 \\ 2 & -3 \end{bmatrix}.

Show the answer

The form is $h_1^2 + 4h_1 h_2 - 3h_2^2$ , with the $4$ coming from the two entries equal to $2$ . At $\langle 2, 1 \rangle$ it equals $4 + 8 - 3 = 9$ .

Use the remainder to show that the linear model $x + 2y$ of $f(x, y) = \sin(x + 2y)$ at the origin is off by at most $0.045$ at $(0.1, 0.1)$ .

Show the answer

The function is $C^2$ on $\mathbb{R}^2$ , and its second partials are $-\sin(x + 2y)$ times $1$ , $2$ and $4$ . The remainder formula with $k = 1$ gives, for some number $c$ between $0$ and $0.3$ ,

R_1 = -\tfrac{1}{2} \sin(c) \, (h_1 + 2h_2)^2.

Since $\lvert \sin c \rvert \le 1$ , the error is at most $\frac{1}{2} (0.3)^2 = 0.045$ . The actual error is $0.3 - \sin 0.3 \approx 0.0045$ .

Multivariable Taylor Approximation

Key terms

Symmetry of mixed partials

Hessian

Quadratic forms

Second-order Taylor polynomial

Multi-index notation

Remainder

Approximation error

Check yourself

How this lesson connects