The Spectral Theorem for Symmetric Matrices

Definition A real $n \times n$ matrix $A$ is symmetric if $A^{\top} = A$, that is, $a_{ij} = a_{ji}$ for all $i$ and $j$. Only a square matrix can be symmetric, and the diagonal entries can be any real numbers. The property used again and again is that a symmetric matrix can move from one side of a dot product to the other. For all $\mathbf{x}$ and $\mathbf{y}$ in $\mathbb{R}^n$,

For every real $m \times n$ matrix $C$, the $n \times n$ matrix $C^{\top}C$ is symmetric, since $(C^{\top}C)^{\top} = C^{\top}C$.

Definition Let $A$ be a real symmetric $n \times n$ matrix. Then every root of the characteristic polynomial $\chi_A(t) = \det(tI - A)$ is real, and $\mathbb{R}^n$ has an orthonormal basis made of eigenvectors of $A$. Equivalently, there are a real orthogonal matrix $Q$, meaning $Q^{\top}Q = I$, and a real diagonal matrix $D$ with

The columns of $Q$ are the orthonormal eigenvectors, and the diagonal entries of $D$ are their eigenvalues in the same order. The hypotheses are real entries, a square symmetric matrix and finite dimension. The first claim, that the eigenvalues are real, has a short proof. Suppose $A\mathbf{v} = \lambda\mathbf{v}$ with $\lambda$ complex and $\mathbf{v}$ a nonzero vector in $\mathbb{C}^n$. Multiplying on the left by $\mathbf{v}^{*} = \overline{\mathbf{v}}^{\top}$ gives

Because $A$ is real and symmetric, $A^{*} = A$, so the number $\mathbf{v}^{*}A\mathbf{v}$ equals its own conjugate $(\mathbf{v}^{*}A\mathbf{v})^{*} = \mathbf{v}^{*}A^{*}\mathbf{v}$ and is real. Also $\mathbf{v}^{*}\mathbf{v} = \lvert v_1 \rvert^2 + \cdots + \lvert v_n \rvert^2 > 0$, and dividing shows that $\lambda$ is real. The second claim, that there are always enough orthonormal eigenvectors for a basis, needs a longer argument by induction on $n$, which this lesson does not give. Neither the examples below nor the orthogonality result for distinct eigenvalues proves it. Linear Algebra II proves it in The Spectral Theorem for Normal Operators.

Definition Let $A$ be a real symmetric $n \times n$ matrix, and let $\mathbf{u}$ and $\mathbf{v}$ in $\mathbb{R}^n$ be eigenvectors with $A\mathbf{u} = \lambda\mathbf{u}$ and $A\mathbf{v} = \mu\mathbf{v}$, where $\lambda \ne \mu$. Then $\mathbf{u} \cdot \mathbf{v} = 0$. The proof uses symmetry in the middle step,

So $(\lambda - \mu)(\mathbf{u} \cdot \mathbf{v}) = 0$, and since $\lambda - \mu \ne 0$, the dot product is $0$. As a result, every vector of the eigenspace $E_{\lambda} = \operatorname{Null}(A - \lambda I)$ is orthogonal to every vector of $E_{\mu}$. If $\lambda = \mu$, the factor $\lambda - \mu$ is $0$ and the computation gives no information about $\mathbf{u} \cdot \mathbf{v}$. Indeed two eigenvectors for one eigenvalue need not be orthogonal, as the repeated eigenvalue example below shows.

Definition Let $A$ be a real $n \times n$ matrix. An orthonormal eigenbasis of $A$ is a basis $\mathcal{B} = (\mathbf{q}_1, \dots, \mathbf{q}_n)$ of $\mathbb{R}^n$ in which every vector is a unit eigenvector, $A\mathbf{q}_i = \lambda_i\mathbf{q}_i$ with $\lVert \mathbf{q}_i \rVert = 1$, and different vectors are orthogonal, $\mathbf{q}_i \cdot \mathbf{q}_j = 0$ for $i \ne j$. The matrix $Q = [\mathbf{q}_1 \ \cdots \ \mathbf{q}_n]$ then satisfies $Q^{\top}Q = I$, so it is orthogonal and $Q^{-1} = Q^{\top}$. The coordinates of a vector in this basis are dot products,

so no linear system has to be solved. The coordinate change keeps lengths, since $\lVert Q\mathbf{z} \rVert^2 = \mathbf{z}^{\top}Q^{\top}Q\mathbf{z} = \lVert \mathbf{z} \rVert^2$. To get unit vectors from perpendicular eigenvectors, divide each by its length. A nonzero multiple of an eigenvector is still an eigenvector for the same eigenvalue.

Definition A real $n \times n$ matrix $A$ is orthogonally diagonalizable if

for a real orthogonal matrix $Q$ and a real diagonal matrix $D$, or equivalently $Q^{\top}AQ = D$. The columns of $Q$ then form an orthonormal eigenbasis, and the diagonal entries of $D$ are the matching eigenvalues in the same order. With columns $\mathbf{q}_1, \dots, \mathbf{q}_n$ and diagonal entries $\lambda_1, \dots, \lambda_n$, the product expands into the spectral decomposition

A real matrix is orthogonally diagonalizable exactly when it is symmetric. The spectral theorem gives one direction. For the other, $(QDQ^{\top})^{\top} = QD^{\top}Q^{\top} = QDQ^{\top}$, so every matrix of this form is symmetric. To orthogonally diagonalize a symmetric matrix, find its eigenvalues, find an orthonormal basis of each eigenspace, and place the basis vectors in $Q$ and their eigenvalues in $D$ in the same order.

Definition Let $\lambda$ be an eigenvalue of a real symmetric matrix $A$ whose eigenspace $E_{\lambda} = \operatorname{Null}(A - \lambda I)$ has dimension $m \ge 2$. Every nonzero vector of $E_{\lambda}$ is an eigenvector for $\lambda$, and every linear combination of vectors in $E_{\lambda}$ stays in $E_{\lambda}$. So Gram-Schmidt applied to any basis of $E_{\lambda}$ produces an orthonormal basis of $E_{\lambda}$ made of eigenvectors. This step is needed in general, because two basis vectors of one eigenspace need not be orthogonal. For a symmetric matrix, $m$ equals the algebraic multiplicity of $\lambda$, a consequence of the spectral theorem. Putting together orthonormal bases of all the eigenspaces gives an orthonormal eigenbasis of $\mathbb{R}^n$. The combined list is orthonormal because different eigenspaces are already orthogonal to each other, and it has $n$ vectors because, by the spectral theorem, the dimensions of the eigenspaces add up to $n$. The result is not unique, since any orthonormal basis of each eigenspace works.