Fourier Series and Orthogonal Expansions

Definition Fix $L > 0$. For real functions $f$ and $g$ that are piecewise continuous on $[-L, L]$, the inner product and the norm are

and $f$ and $g$ are orthogonal when $\langle f, g \rangle = 0$. Write $c_n(x) = \cos(n\pi x/L)$ for $n \ge 0$, so that $c_0 = 1$, and $s_n(x) = \sin(n\pi x/L)$ for $n \ge 1$. All of these functions have period $2L$, and they are mutually orthogonal. For $m \ne n$,

and $\langle s_m, c_n \rangle = 0$ for all $m \ge 1$ and $n \ge 0$. The squared norms are

for $n \ge 1$. For two cosines or two sines, a product-to-sum identity turns the product into cosines with angular frequencies $(m - n)\pi/L$ and $(m + n)\pi/L$, and a cosine whose frequency is a nonzero integer multiple of $\pi/L$ integrates to zero over $[-L, L]$. A sine times a cosine is an odd function, so its integral over the symmetric interval is zero. The identities $\sin^2\theta = \tfrac{1}{2}(1 - \cos 2\theta)$ and $\cos^2\theta = \tfrac{1}{2}(1 + \cos 2\theta)$ give the squared norm $L$.

Definition The Fourier coefficients of a function $f$ that is piecewise continuous on $[-L, L]$ are

for $n = 0, 1, 2, \dots$ and

for $n = 1, 2, 3, \dots$. In the notation above, $a_n = \langle f, c_n \rangle / L$ and $b_n = \langle f, s_n \rangle / L$. The factor $1/L$ comes from orthogonality. Suppose $f = A_0 + \sum_{n=1}^{N}(A_n c_n + B_n s_n)$ is a finite combination. Taking the inner product with $c_m$ for some $1 \le m \le N$ removes every term but one and leaves $\langle f, c_m \rangle = A_m L$, so $A_m = a_m$. In the same way $B_m = b_m$. The constant has the larger squared norm $2L$, so $\langle f, 1 \rangle = 2L A_0$ and the constant amplitude is $A_0 = a_0/2$. This is why the series below begins with $a_0/2$, and why one formula serves every $n \ge 0$. Some books call the constant amplitude itself $a_0$ and put $1/(2L)$ in its formula. Either convention works as long as the coefficient formulas and the series are kept together.

Symmetry saves work. If $f$ is odd, every $a_n$ is zero and $b_n$ is $2/L$ times the integral of $f(x)\sin(n\pi x/L)$ over $[0, L]$. If $f$ is even, every $b_n$ is zero and $a_n$ is $2/L$ times the integral of $f(x)\cos(n\pi x/L)$ over $[0, L]$.

Definition The Fourier series of a function $f$ that is piecewise continuous on $[-L, L]$ is

with the coefficients defined above. Its $N$th partial sum $S_N f$ keeps the terms with $n \le N$. Every term has period $2L$, so the series describes the $2L$-periodic extension of $f$, the function on the whole line that repeats the values of $f$ on $(-L, L)$. The sign $\sim$ only attaches the series to $f$. It does not claim that the series converges, or that its sum is $f(x)$. Where it converges and to what is a separate question, which the next lesson answers for piecewise smooth functions.

Definition Let $T_N$ be the set of trigonometric polynomials of degree at most $N$ with period $2L$, the functions

with real constants $A_n$ and $B_n$. The orthogonal projection of a piecewise continuous $f$ onto $T_N$ is the function $Pf$ in $T_N$ for which the residual $f - Pf$ is orthogonal to every element of $T_N$, or equivalently to $1$ and to $c_n$ and $s_n$ for $1 \le n \le N$. It is unique. If two members of $T_N$ had this property, their difference would lie in $T_N$ and be orthogonal to itself, so it would have norm zero, and the only trigonometric polynomial with norm zero is the zero function. Because the basis is orthogonal, the projection is found one direction at a time. The amplitude of each basis function $\varphi$ is $\langle f, \varphi \rangle / \lVert \varphi \rVert^2$, which is $a_0/2$ for the constant, $a_n$ for $c_n$ and $b_n$ for $s_n$. Hence

the $N$th partial sum of the Fourier series. The spaces are nested, $T_N \subseteq T_{N+1}$, so a projection with more modes keeps all the old amplitudes and adds new ones.

Definition Let $f$ be piecewise continuous on $[-L, L]$ and let $N \ge 0$. The mean-square error of a trigonometric polynomial $T$ in $T_N$ is

which is $\lVert f - T \rVert^2$. The partial sum $S_N f$ is the best mean-square approximation of $f$ in $T_N$, meaning $E(T) > E(S_N f)$ for every other $T$ in $T_N$. The reason is the Pythagorean theorem. The residual $f - S_N f$ is orthogonal to $S_N f - T$, which lies in $T_N$, so

and the last term is positive unless $T = S_N f$. The smallest error is

Since an error is never negative, this gives Bessel’s inequality $Q_N \le \lVert f \rVert^2 / L$ for every $N$. Because $T_N \subseteq T_{N+1}$, the smallest error can only decrease as $N$ grows. All of this concerns the integral of the squared error. It does not say that $S_N f$ makes the largest pointwise error as small as possible.

Definition If $f$ is piecewise continuous on $[-L, L]$, then

The same holds for every square-integrable $f$, with the integral taken in the Lebesgue sense. By the formula for the smallest error, the identity is equivalent to $E(S_N f) \to 0$ as $N \to \infty$, so the partial sums converge to $f$ in the mean-square sense. It expresses the completeness of the sines and cosines. Bessel’s inequality becomes an equality in the limit, and a piecewise continuous function whose Fourier coefficients are all zero vanishes at every point where it is continuous. The identity says nothing about single points. Changing $f$ at finitely many points changes no integral, so it changes no coefficient and neither side of the identity, and mean-square convergence alone does not decide the sum of the series at any particular point.