Invariant Subspaces and Block Matrices, linear algebra II lesson

Apply the real matrix with rows $(2, 1, 0)$ , $(0, 3, 0)$ and $(0, 0, 4)$ to the standard basis vectors. The first one stays on its own axis, but the second one picks up a component along the first axis. So the second coordinate feeds into the first, while nothing flows back. This lesson explains how a proper nonzero subspace that an operator maps into itself produces a block of zeros in its matrix once a basis of that subspace is listed first, when a second zero block appears, and why the orthogonal complement of such a subspace can still fail to be mapped into itself.

Key terms

Invariant subspace

Definition Let $V$ be a vector space over $\mathbb{F}$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , and let $T \in \operatorname{Hom}(V, V)$ be an operator on $V$ . A subspace $U$ of $V$ is invariant under $T$ , or $T$ -invariant, if $Tu \in U$ for every $u \in U$ , that is,

T(U) \subseteq U.

If $U = \operatorname{span}\{u_1, \dots, u_k\}$ , it is enough to check that $Tu_1, \dots, Tu_k$ lie in $U$ , because

\begin{aligned} &T(c_1u_1 + \cdots + c_ku_k) \\[0.4em] &\quad = c_1Tu_1 + \cdots + c_kTu_k \end{aligned}

and $U$ is closed under linear combinations. The subspaces $\{0\}$ , $V$ , $\ker T$ and $\operatorname{im} T$ are always invariant, and so is every eigenspace of $T$ . A line $\operatorname{span}\{v\}$ with $v \ne 0$ is invariant exactly when $v$ is an eigenvector of $T$ . Invariance does not require $T$ to fix the vectors of $U$ , and it does not require them to be eigenvectors.

In plain words The operator may stretch, turn or collapse the vectors of $U$ , but it never pushes one of them out of $U$ .

Example Tuples such as $(1, 2, 0)$ stand for column vectors. Let

A = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 4 \end{bmatrix}

act on $\mathbb{R}^3$ . Since $Ae_1 = 2e_1$ , the line $\operatorname{span}\{e_1\}$ is invariant. The line $\operatorname{span}\{e_2\}$ is not, because $Ae_2 = e_1 + 3e_2$ . The plane $\operatorname{span}\{e_1, e_2\}$ is invariant, since $Ae_1$ and $Ae_2$ both lie in it, and yet it contains vectors that are not eigenvectors. For instance $A(1, 2, 0) = (4, 6, 0)$ , which is not a multiple of $(1, 2, 0)$ .

Restriction of an operator

Definition Let $V$ be a vector space over $\mathbb{F}$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , let $T \in \operatorname{Hom}(V, V)$ , and let $U$ be a $T$ -invariant subspace. The restriction of $T$ to $U$ is the operator $T|_U \in \operatorname{Hom}(U, U)$ given by

T|_U(u) = Tu, \qquad u \in U.

It follows the same rule as $T$ , but its domain and codomain are both $U$ . Invariance is exactly what allows $U$ as the codomain. Without it, the rule only defines a map from $U$ to $V$ . Every eigenvector of $T|_U$ is an eigenvector of $T$ with the same eigenvalue. If $\mathbb{F} = \mathbb{C}$ and $U$ is finite-dimensional and nonzero, then $T|_U$ has an eigenvalue, by the fundamental theorem of algebra applied to its characteristic polynomial. So every nonzero finite-dimensional invariant subspace of a complex operator contains an eigenvector of $T$ . Over $\mathbb{R}$ this can fail.

In plain words Once $T$ keeps $U$ to itself, you can ignore the rest of $V$ and study $T$ as an operator on the smaller space $U$ .

Example For the matrix $A$ above and $U = \operatorname{span}\{e_1, e_2\}$ , the images $Ae_1 = 2e_1$ and $Ae_2 = e_1 + 3e_2$ give the matrix of $A|_U$ in the basis $(e_1, e_2)$ ,

[A|_U] = \begin{bmatrix} 2 & 1 \\ 0 & 3 \end{bmatrix}.

Its eigenvalues $2$ and $3$ are eigenvalues of $A$ , with eigenvectors $e_1$ and $(1, 1, 0)$ that lie in $U$ . There is no restriction of $A$ to $\operatorname{span}\{e_2, e_3\}$ as an operator, because $Ae_2$ leaves that plane. The real quarter turn with rows $(0, -1)$ and $(1, 0)$ shows the failure over $\mathbb{R}$ . The whole plane $\mathbb{R}^2$ is invariant and nonzero, yet it contains no real eigenvector.

Adapted basis

Definition Let $V$ be a vector space over $\mathbb{F}$ of finite dimension $n$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , and let $U$ be a subspace of dimension $r$ . A basis

\mathcal{B} = (u_1, \dots, u_r, w_1, \dots, w_{n-r})

of $V$ is adapted to $U$ if its first $r$ vectors $u_1, \dots, u_r$ form a basis of $U$ . One always exists, since every basis of $U$ extends to a basis of $V$ . The remaining vectors span a complement $W = \operatorname{span}\{w_1, \dots, w_{n-r}\}$ , so $V = U \oplus W$ . When $U$ is neither $\{0\}$ nor $V$ , different extensions can give different complements. A vector lies in $U$ exactly when its last $n - r$ coordinates in $\mathcal{B}$ are zero. More generally, a basis is adapted to a direct sum $V = U_1 \oplus \cdots \oplus U_k$ when it lists a basis of $U_1$ , then a basis of $U_2$ , and so on.

In plain words List a basis of the subspace first. Then membership in the subspace becomes a visible pattern of zeros in the coordinates, and invariance becomes a pattern of zeros in the matrix.

Example In $\mathbb{R}^2$ let $U = \operatorname{span}\{(1, 1)\}$ and take $b_1 = (1, 1)$ and $b_2 = (0, 1)$ . Writing $(v_1, v_2) = c_1b_1 + c_2b_2$ gives

c_1 = v_1, \qquad c_2 = v_2 - v_1.

So the second coordinate vanishes exactly when $v_1 = v_2$ , that is, exactly on $U$ . For instance $(3, 3)$ has coordinates $(3, 0)$ , while $(2, 5)$ has coordinates $(2, 3)$ . Choosing $b_2 = (1, 0)$ instead gives another basis adapted to $U$ , with a different complement.

Block upper triangular matrix

Definition Let $V$ be a vector space over $\mathbb{F}$ of finite dimension $n$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , let $T \in \operatorname{Hom}(V, V)$ , and let $\mathcal{B} = (u_1, \dots, u_r, w_1, \dots, w_{n-r})$ be a basis adapted to a subspace $U$ of dimension $r$ , with $0 < r < n$ . Write $[T]_{\mathcal{B}}$ for $[T]_{\mathcal{B} \leftarrow \mathcal{B}}$ , whose $j$ th column holds the coordinates of the image of the $j$ th basis vector. Then $U$ is $T$ -invariant if and only if

[T]_{\mathcal{B}} = \begin{bmatrix} B & C \\ 0 & D \end{bmatrix},

with $B \in \mathbb{F}^{r \times r}$ , $C \in \mathbb{F}^{r \times (n-r)}$ , $D \in \mathbb{F}^{(n-r) \times (n-r)}$ and a zero block of size $(n - r) \times r$ . A matrix of this shape is block upper triangular. The reason is that the first $r$ columns hold the coordinates of $Tu_1, \dots, Tu_r$ , and their last $n - r$ entries vanish exactly when every $Tu_j$ lies in $U$ .

When $U$ is invariant, $B$ is the matrix of $T|_U$ in the basis $(u_1, \dots, u_r)$ , and $C$ holds the $U$ components of $Tw_1, \dots, Tw_{n-r}$ . The block $D$ is the matrix of the operator $w \mapsto P(Tw)$ on $W = \operatorname{span}\{w_1, \dots, w_{n-r}\}$ , where $P$ is the projection onto $W$ along $U$ . When $W$ is invariant as well, $D$ is the matrix of $T|_W$ . Otherwise $T$ does not restrict to an operator on $W$ . A vector whose coordinates split into blocks $c \in \mathbb{F}^r$ and $d \in \mathbb{F}^{n-r}$ is sent to the vector with blocks $Bc + Cd$ and $Dd$ .

The characteristic polynomial $\chi_T(t) = \det(tI - [T]_{\mathcal{B}})$ factors even when $C \ne 0$ ,

\chi_T(t) = \chi_B(t)\,\chi_D(t).

Indeed, $tI - [T]_{\mathcal{B}}$ is again block upper triangular. In the permutation expansion of its determinant, every term that uses an entry of the zero block vanishes. A surviving term must send the last $n - r$ rows to the last $n - r$ columns, and hence the first $r$ rows to the first $r$ columns, so the sum of these terms factors. In particular $\det T = \det B \det D$ .

In plain words The zero block says that nothing in $U$ produces a component outside $U$ . The block $C$ still allows influence in the other direction, from the complement into $U$ , so the coupling runs one way.

Example Let

A = \begin{bmatrix} 3 & -1 \\ 1 & 1 \end{bmatrix}.

Since $A(1, 1) = (2, 2)$ , the line $U = \operatorname{span}\{(1, 1)\}$ is invariant. In the adapted basis $b_1 = (1, 1)$ , $b_2 = (0, 1)$ we get $Ab_1 = 2b_1$ and $Ab_2 = (-1, 1) = -b_1 + 2b_2$ , so

\begin{gathered} [A]_{\mathcal{B}} = \begin{bmatrix} 2 & -1 \\ 0 & 2 \end{bmatrix}, \\[0.4em] \chi_A(t) = (t - 2)^2. \end{gathered}

Here $B = [2]$ , $C = [-1]$ and $D = [2]$ . No choice of complement makes $C$ zero. An invariant complement would be another invariant line, hence another eigenline, but $A - 2I$ has rank $1$ , so the only eigenline is $U$ itself.

Block diagonal matrix

Definition Let $V$ be a finite-dimensional vector space over $\mathbb{F}$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , let $T \in \operatorname{Hom}(V, V)$ , and let $V = U \oplus W$ with $U$ and $W$ nonzero. In a basis $\mathcal{B}$ that lists a basis of $U$ and then a basis of $W$ ,

[T]_{\mathcal{B}} = \begin{bmatrix} B & 0 \\ 0 & D \end{bmatrix}

if and only if both $U$ and $W$ are $T$ -invariant. A matrix of this shape is block diagonal. The lower left block vanishes exactly when $U$ is invariant, and the upper right block vanishes exactly when every basis vector of $W$ is sent into $W$ . In that case $B$ and $D$ are the matrices of $T|_U$ and $T|_W$ , and

\chi_T = \chi_{T|_U}\,\chi_{T|_W}.

The same holds for $V = U_1 \oplus \cdots \oplus U_k$ , where invariance of every $U_i$ is equivalent to a block diagonal matrix with one block for each $U_i$ . The blocks themselves need not be diagonal. Invariance of $U$ alone gives only block upper triangular form. An invariant complement may fail to exist, and when one exists it need not be orthogonal to $U$ .

In plain words Block diagonal means the operator splits into independent pieces, one on each subspace, with no influence between them in either direction.

Example For the $3 \times 3$ matrix $A$ with rows $(2, 1, 0)$ , $(0, 3, 0)$ and $(0, 0, 4)$ , the plane $\operatorname{span}\{e_1, e_2\}$ and the line $\operatorname{span}\{e_3\}$ are both invariant, so the standard basis already gives the two blocks

\begin{bmatrix} 2 & 1 \\ 0 & 3 \end{bmatrix} \quad \text{and} \quad \begin{bmatrix} 4 \end{bmatrix}.

The first block is not diagonal, because $e_2$ still feeds into $e_1$ inside the plane. The line $\operatorname{span}\{e_1\}$ also has an invariant complement, the plane spanned by $(1, 1, 0)$ and $e_3$ , since $A(1, 1, 0) = (3, 3, 0)$ and $Ae_3 = 4e_3$ . In the basis $e_1$ , $(1, 1, 0)$ , $e_3$ the matrix is $\operatorname{diag}(2, 3, 4)$ . That complement is not perpendicular to $e_1$ , since $e_1 \cdot (1, 1, 0) = 1$ .

Reducing subspace

Definition Let $V$ be a finite-dimensional inner product space over $\mathbb{F}$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ , let $T \in \operatorname{Hom}(V, V)$ with adjoint $T^{*}$ , and let $U$ be a subspace of $V$ . The subspace $U$ reduces $T$ , or is a reducing subspace for $T$ , if both $U$ and $U^{\perp}$ are $T$ -invariant. The key fact is that if $U$ is $T$ -invariant, then $U^{\perp}$ is $T^{*}$ -invariant. Indeed, for $w \in U^{\perp}$ and every $u \in U$ ,

\langle u, T^{*}w \rangle = \langle Tu, w \rangle = 0,

because $Tu \in U$ , so $T^{*}w \in U^{\perp}$ . Applying this fact to the pair $U^{\perp}$ , $T$ and to the pair $U$ , $T^{*}$ , and using $(U^{\perp})^{\perp} = U$ and $(T^{*})^{*} = T$ , gives the test

\begin{gathered} U \text{ reduces } T \\[0.4em] \iff U \text{ is invariant} \\[0.4em] \text{under } T \text{ and } T^{*}. \end{gathered}

Since $V = U \oplus U^{\perp}$ , an orthonormal basis of $U$ followed by an orthonormal basis of $U^{\perp}$ is adapted to this sum, and $U$ reduces $T$ exactly when the matrix of $T$ in that basis is block diagonal. If $T$ is self-adjoint, so that $T = T^{*}$ , every invariant subspace reduces $T$ .

In plain words An invariant subspace blocks influence in one direction. A reducing subspace blocks it in both, and the adjoint controls the second direction. Self-adjoint operators get the second direction for free.

Example Use the dot product on $\mathbb{R}^3$ , so the adjoint of the matrix $A$ with rows $(2, 1, 0)$ , $(0, 3, 0)$ , $(0, 0, 4)$ is $A^{\mathsf{T}}$ . The line $\operatorname{span}\{e_1\}$ is invariant under $A$ but does not reduce it, since $A^{\mathsf{T}}e_1 = (2, 1, 0)$ leaves the line. Equivalently, its orthogonal complement $\operatorname{span}\{e_2, e_3\}$ is not invariant, because $Ae_2 = e_1 + 3e_2$ . The plane $\operatorname{span}\{e_1, e_2\}$ does reduce $A$ , since $A^{\mathsf{T}}e_1 = (2, 1, 0)$ and $A^{\mathsf{T}}e_2 = (0, 3, 0)$ stay in it, and indeed its complement $\operatorname{span}\{e_3\}$ is invariant. For the symmetric matrix with rows $(2, 1)$ and $(1, 2)$ , the invariant line through $(1, 1)$ has the invariant orthogonal complement through $(1, -1)$ , as the self-adjoint case predicts, since $(1, -1)$ is sent to itself.

Check yourself

Let $A$ be the real matrix with rows $(4, -1)$ and $(2, 1)$ . Is the line through $(1, 1)$ invariant under $A$ ? Is the line through $(1, 0)$ ? Write the matrix of $A$ in the basis $b_1 = (1, 1)$ , $b_2 = (0, 1)$ .

Show the answer

Since $A(1, 1) = (3, 3) = 3(1, 1)$ , the line through $(1, 1)$ is invariant. Since $A(1, 0) = (4, 2)$ is not a multiple of $(1, 0)$ , the line through $(1, 0)$ is not. Next, $Ab_2 = (-1, 1) = -b_1 + 2b_2$ , so

[A]_{\mathcal{B}} = \begin{bmatrix} 3 & -1 \\ 0 & 2 \end{bmatrix}.

The zero below the diagonal records the invariant line. As a check, $(t - 3)(t - 2) = t^2 - 5t + 6$ , and indeed $A$ has trace $5$ and determinant $6$ .

Does the real quarter turn $R$ with rows $(0, -1)$ and $(1, 0)$ have an invariant line in $\mathbb{R}^2$ ? Does it have an invariant line in $\mathbb{C}^2$ when it acts on complex vectors?

Show the answer

Not in $\mathbb{R}^2$ . A real invariant line would be spanned by a real $v \ne 0$ with $Rv = cv$ for some real $c$ . Applying $R$ twice gives $R^2v = c^2v$ , but $R^2 = -I$ , so $c^2 v = -v$ and $c^2 = -1$ , which no real number satisfies. The only real invariant subspaces are $\{0\}$ and $\mathbb{R}^2$ .

In $\mathbb{C}^2$ it does. The vectors $(1, -i)$ and $(1, i)$ satisfy $R(1, -i) = i(1, -i)$ and $R(1, i) = -i(1, i)$ , so the complex lines they span are invariant. Which subspaces are invariant depends on the field.

Let $A$ be the real matrix with rows $(1, 1, 0)$ , $(1, 1, 0)$ and $(0, 2, 5)$ , and use the dot product on $\mathbb{R}^3$ . Show that $U = \operatorname{span}\{e_3\}$ is invariant, decide whether it reduces $A$ , and write the matrix of $A$ in the orthonormal basis $\mathcal{B} = (e_3, e_1, e_2)$ .

Show the answer

Since $Ae_3 = 5e_3$ , the line $U$ is invariant. Its orthogonal complement is $\operatorname{span}\{e_1, e_2\}$ . The image $Ae_1 = (1, 1, 0)$ stays in it, but $Ae_2 = (1, 1, 2)$ does not, so $U$ does not reduce $A$ . The adjoint test agrees, since $A^{\mathsf{T}}e_3 = (0, 2, 5)$ is not a multiple of $e_3$ .

In the basis $(e_3, e_1, e_2)$ the images of the basis vectors are $5e_3$ , then $e_1 + e_2$ , then $2e_3 + e_1 + e_2$ , so

[A]_{\mathcal{B}} = \begin{bmatrix} 5 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 1 & 1 \end{bmatrix}.

This is block upper triangular with $B = [5]$ , $C = \begin{bmatrix} 0 & 2 \end{bmatrix}$ and $D$ the block with rows $(1, 1)$ and $(1, 1)$ , whose characteristic polynomial is $t^2 - 2t$ . Hence $\chi_A(t) = t(t - 2)(t - 5)$ . The line does have an invariant complement, spanned by the eigenvectors $(5, -5, 2)$ and $(3, 3, -2)$ for the eigenvalues $0$ and $2$ , but that complement is not $U^{\perp}$ .

Invariant Subspaces and Block Matrices

Key terms

Invariant subspace

Restriction of an operator

Adapted basis

Block upper triangular matrix

Block diagonal matrix

Reducing subspace

Check yourself

How this lesson connects

Builds on

Used later in