Unit 8, lesson 43

Generalized Eigenspace Decomposition

Why the space is the direct sum of the generalized eigenspaces of an operator whose minimal polynomial splits, proved with polynomial projectors, with nilpotent restrictions and dimensions equal to algebraic multiplicities.

  • Video 43
  • Chains and Generalized Eigenspaces

The matrix with rows (1,1,1)(1, 1, 1), (0,1,1)(0, 1, 1) and (0,0,2)(0, 0, 2) has the eigenvalues 11 and 22 but only two independent eigenvectors, so it cannot be diagonalized. Its action still separates into a plane where only the eigenvalue 11 matters and a line spanned by (2,1,1)(2, 1, 1) where it multiplies by 22, even though that line cannot be read off from the entries. This lesson proves that such a separation always exists when the minimal polynomial splits into linear factors, using projections that are polynomials in the operator, and shows that the pieces are generalized eigenspaces whose dimensions are the algebraic multiplicities. Throughout, F\mathbb{F} is R\mathbb{R} or C\mathbb{C}, VV is a vector space over F\mathbb{F} of finite dimension n≥1n \ge 1, T ⁣:V→VT \colon V \to V is linear, a square matrix AA stands for the operator x↦Axx \mapsto Ax, and a tuple such as (2,1,1)(2, 1, 1) stands for a column vector.

Key terms

Split minimal polynomial

Definition A polynomial with coefficients in F\mathbb{F} splits over F\mathbb{F} if it is a constant times a product of linear factors t−ct - c with every c∈Fc \in \mathbb{F}. The minimal polynomial of TT splits over F\mathbb{F} when

μT(t)=(t−λ1)s1⋯(t−λr)sr\mu_T(t) = (t - \lambda_1)^{s_1} \cdots (t - \lambda_r)^{s_r}

with distinct λ1,…,λr∈F\lambda_1, \dots, \lambda_r \in \mathbb{F} and exponents sj≥1s_j \ge 1. The roots of μT\mu_T in F\mathbb{F} are exactly the eigenvalues of TT, so the λj\lambda_j are then all the eigenvalues of TT. Over C\mathbb{C} every polynomial splits, by the fundamental theorem of algebra, so the hypothesis holds for every complex operator. Over R\mathbb{R} it holds exactly when every complex root of μT\mu_T is real. The factors aj=(t−λj)sja_j = (t - \lambda_j)^{s_j} are pairwise coprime, even when some exponents exceed 11, because no two of them have a common root.

In plain words Splitting means that the minimal polynomial factors completely with scalars from the field you are working in. It is the hypothesis that lets every vector be sorted by eigenvalue.

Example Let

A=[111011002].A = \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 2 \end{bmatrix}.

Its characteristic polynomial is (t−1)2(t−2)(t - 1)^2(t - 2), and μA\mu_A divides it and has both 11 and 22 as roots. The product (A−I)(A−2I)(A - I)(A - 2I) is not zero, while (A−I)2(A−2I)=0(A - I)^2(A - 2I) = 0, so μA(t)=(t−1)2(t−2)\mu_A(t) = (t - 1)^2(t - 2), which splits over R\mathbb{R}. By contrast, the quarter turn with rows (0,−1)(0, -1) and (1,0)(1, 0) has minimal polynomial t2+1t^2 + 1. It has no real root, so it does not split over R\mathbb{R}, but over C\mathbb{C} it splits as (t−i)(t+i)(t - i)(t + i).

Primary decomposition

Definition Let μT=a1a2⋯ar\mu_T = a_1 a_2 \cdots a_r with monic polynomials aja_j of positive degree that are pairwise coprime, and let bjb_j be the product of all the factors except aja_j. Each bjb_j is coprime to aja_j, so Bézout’s identity gives polynomials uj,vju_j, v_j with ujaj+vjbj=1u_j a_j + v_j b_j = 1. Put hj=vjbjh_j = v_j b_j and Pj=hj(T)P_j = h_j(T), and let Kj=ker⁡aj(T)K_j = \ker a_j(T). Then

P1+⋯+Pr=I,Pj2=Pj,PjPk=0  for j≠k,\begin{gathered} P_1 + \cdots + P_r = I, \\[0.4em] P_j^2 = P_j, \qquad P_j P_k = 0 \ \text{ for } j \ne k, \end{gathered}

the image of PjP_j is KjK_j, each KjK_j is invariant under TT, and

V=K1⊕K2⊕⋯⊕Kr,V = K_1 \oplus K_2 \oplus \cdots \oplus K_r,

with PjP_j the projection onto KjK_j along the sum of the other pieces. When the aja_j are the powers of the distinct monic irreducible factors of μT\mu_T, this is the primary decomposition of VV under TT. When μT\mu_T splits, those factors are aj=(t−λj)sja_j = (t - \lambda_j)^{s_j}.

For the proof, note that hj=1−ujajh_j = 1 - u_j a_j leaves remainder 11 on division by aja_j, while hkh_k for k≠jk \ne j is a multiple of aja_j, because aja_j is a factor of bkb_k. So every aja_j divides 1−(h1+⋯+hr)1 - (h_1 + \cdots + h_r). Pairwise coprime polynomials that each divide ff have a product that divides ff. For two of them, aa and bb with ua+vb=1ua + vb = 1, write f=agf = ag. Then g=uag+vbgg = uag + vbg, and bb divides both terms because ag=fag = f, so bb divides gg and abab divides ff. Induction handles more factors. Hence μT\mu_T divides 1−∑hk1 - \sum h_k, and evaluating at TT gives ∑Pk=I\sum P_k = I. For j≠kj \ne k the product bjbkb_j b_k contains every aia_i, so μT\mu_T divides hjhkh_j h_k and PjPk=0P_j P_k = 0. Multiplying ∑Pk=I\sum P_k = I by PjP_j gives Pj2=PjP_j^2 = P_j. Since ajbj=μTa_j b_j = \mu_T, we get aj(T)Pj=vj(T)μT(T)=0a_j(T) P_j = v_j(T) \mu_T(T) = 0, so the image of PjP_j lies in KjK_j, and on KjK_j the operator Pj=I−uj(T)aj(T)P_j = I - u_j(T) a_j(T) is the identity. Every x∈Vx \in V equals P1x+⋯+PrxP_1 x + \cdots + P_r x with Pkx∈KkP_k x \in K_k, so the pieces span VV. Finally, if w1+⋯+wr=0w_1 + \cdots + w_r = 0 with wk∈Kkw_k \in K_k, then PiP_i fixes wiw_i and kills every other wkw_k, since aka_k is a factor of bib_i. Applying PiP_i gives wi=0w_i = 0, so the sum is direct. Each KjK_j is invariant because aj(T)a_j(T) commutes with TT.

In plain words Factor the minimal polynomial into pieces with no common complex root. Bézout’s identity turns that factorization into projections that are polynomials in TT. They add up to the identity, and each one keeps exactly the vectors killed by its own factor. Over R\mathbb{R} an irreducible quadratic factor still produces an invariant piece, but that piece contains no real eigenvector.

Example For the matrix AA above take a1=(t−1)2a_1 = (t - 1)^2 and a2=t−2a_2 = t - 2. The identity (t−1)2−t(t−2)=1(t - 1)^2 - t(t - 2) = 1 is a Bézout relation, so h2=(t−1)2h_2 = (t - 1)^2 and h1=1−(t−1)2=−t(t−2)h_1 = 1 - (t - 1)^2 = -t(t - 2). Then

P2=(A−I)2=[002001001],P1=I−P2=[10−201−1000].\begin{aligned} P_2 &= (A - I)^2 = \begin{bmatrix} 0 & 0 & 2 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \end{bmatrix}, \\[0.4em] P_1 &= I - P_2 = \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{bmatrix}. \end{aligned}

The image of P2P_2 is the line spanned by (2,1,1)(2, 1, 1), and A(2,1,1)=(4,2,2)A(2, 1, 1) = (4, 2, 2), so that line consists of eigenvectors for 22. The image of P1P_1 is the xyxy-plane, which is ker⁡(A−I)2\ker (A - I)^2. For e3e_3 the split is e3=(−2,−1,0)+(2,1,1)e_3 = (-2, -1, 0) + (2, 1, 1). So P1P_1 projects onto the xyxy-plane along the line spanned by (2,1,1)(2, 1, 1), not along the zz-axis.

Direct sum of generalized eigenspaces

Definition Suppose μT\mu_T splits over F\mathbb{F} as above. Then for each jj

ker⁡(T−λjI)sj=Gλj(T),\ker (T - \lambda_j I)^{s_j} = G_{\lambda_j}(T),

where Gλ(T)=ker⁡(T−λI)nG_{\lambda}(T) = \ker (T - \lambda I)^n is the generalized eigenspace, and

V=Gλ1(T)⊕⋯⊕Gλr(T).V = G_{\lambda_1}(T) \oplus \cdots \oplus G_{\lambda_r}(T).

Each Gλj(T)G_{\lambda_j}(T) is invariant under TT, and the projection onto it along the others is the polynomial projector PjP_j. The operator need not be diagonalizable. The splitting hypothesis cannot be dropped. If VV is the sum of the generalized eigenspaces for some eigenvalues λ1,…,λr\lambda_1, \dots, \lambda_r in F\mathbb{F}, then ∏j(t−λj)n\prod_j (t - \lambda_j)^n annihilates TT, so μT\mu_T divides it and splits.

For the proof write Kj=ker⁡(T−λjI)sjK_j = \ker (T - \lambda_j I)^{s_j}, so that V=K1⊕⋯⊕KrV = K_1 \oplus \cdots \oplus K_r by the primary decomposition. Since μT\mu_T divides χT\chi_T, which has degree nn, we have sj≤ns_j \le n and Kj⊆Gλj(T)K_j \subseteq G_{\lambda_j}(T). For k≠jk \ne j the operator T−λjIT - \lambda_j I is injective on KkK_k. Indeed, if w∈Kkw \in K_k and Tw=λjwTw = \lambda_j w, then 0=(T−λkI)skw=(λj−λk)skw0 = (T - \lambda_k I)^{s_k} w = (\lambda_j - \lambda_k)^{s_k} w, so w=0w = 0. Now let x∈Gλj(T)x \in G_{\lambda_j}(T) and write x=x1+⋯+xrx = x_1 + \cdots + x_r with xk=Pkx∈Kkx_k = P_k x \in K_k. Then

0=(T−λjI)nx=∑k=1r(T−λjI)nxk,\begin{aligned} 0 &= (T - \lambda_j I)^n x \\[0.4em] &= \sum_{k=1}^{r} (T - \lambda_j I)^n x_k, \end{aligned}

and each term lies in KkK_k by invariance. The sum is direct, so every term is zero. For k≠jk \ne j the power (T−λjI)n(T - \lambda_j I)^n is injective on KkK_k, so xk=0x_k = 0. Hence x=xj∈Kjx = x_j \in K_j.

In plain words When the minimal polynomial splits, every vector is a sum of one vector from each generalized eigenspace, and these pieces are unique. Some pieces may be zero. This works even when there are too few ordinary eigenvectors, because the pieces are generalized eigenspaces rather than eigenspaces.

Example For the matrix AA above, the kernels of (A−I)k(A - I)^k for k=1,2,3k = 1, 2, 3 have dimensions 1,2,21, 2, 2, and every kernel of a positive power of A−2IA - 2I has dimension 11. So

G1(A)=span⁡(e1,e2),G2(A)=span⁡((2,1,1)),\begin{gathered} G_1(A) = \operatorname{span}(e_1, e_2), \\[0.4em] G_2(A) = \operatorname{span}\bigl((2, 1, 1)\bigr), \end{gathered}

while the eigenspace for 11 is only the line spanned by e1e_1. The vectors e1e_1, e2e_2 and (2,1,1)(2, 1, 1) form a basis, so R3=G1(A)⊕G2(A)\mathbb{R}^3 = G_1(A) \oplus G_2(A). For x=(3,2,1)x = (3, 2, 1) the projectors give x=(1,1,0)+(2,1,1)x = (1, 1, 0) + (2, 1, 1). The first piece is a generalized eigenvector of rank 22, since (A−I)(1,1,0)=(1,0,0)≠0(A - I)(1, 1, 0) = (1, 0, 0) \ne 0 and (A−I)2(1,1,0)=0(A - I)^2 (1, 1, 0) = 0.

Nilpotent operator

Definition A linear operator NN on VV is nilpotent if Nk=0N^k = 0 for some positive integer kk, and the smallest such kk is its index of nilpotency. On a space of dimension d≥1d \ge 1 a nilpotent operator satisfies Nd=0N^d = 0, because the kernels of its powers stabilize by step dd and must eventually be the whole space. Its only eigenvalue is 00. Zero is an eigenvalue because an injective NN would have injective powers, yet Nk=0N^k = 0 is not injective. No other scalar is an eigenvalue, since Nw=αwNw = \alpha w with w≠0w \ne 0 gives 0=Nkw=αkw0 = N^k w = \alpha^k w, so α=0\alpha = 0. When μT\mu_T splits as above, the restriction NjN_j of T−λjIT - \lambda_j I to Gλj(T)G_{\lambda_j}(T) is nilpotent with index exactly sjs_j, so on that subspace

T=λjI+Nj.T = \lambda_j I + N_j.

The power NjsjN_j^{s_j} is zero because Gλj(T)=ker⁡(T−λjI)sjG_{\lambda_j}(T) = \ker (T - \lambda_j I)^{s_j}. If Njr=0N_j^{r} = 0 for some r<sjr < s_j, let qq be μT\mu_T with the factor (t−λj)sj(t - \lambda_j)^{s_j} replaced by (t−λj)r(t - \lambda_j)^{r}. Then q(T)q(T) kills Gλj(T)G_{\lambda_j}(T) by assumption and every other Gλk(T)G_{\lambda_k}(T) through its own factor. These subspaces span VV, so q(T)=0q(T) = 0 with deg⁡q<deg⁡μT\deg q < \deg \mu_T, which contradicts minimality. So the restriction of TT has minimal polynomial (t−λj)sj(t - \lambda_j)^{s_j}, and sjs_j is the largest rank of a generalized eigenvector for λj\lambda_j.

In plain words A nilpotent operator gives zero after finitely many applications. On each generalized eigenspace, TT is the scalar λj\lambda_j plus such an operator, and the exponent of t−λjt - \lambda_j in the minimal polynomial says how many applications that takes.

Example For the matrix AA above, A−IA - I sends e1e_1 to 00 and e2e_2 to e1e_1, so its restriction to G1(A)G_1(A) has the matrix

[0100]\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}

in the basis (e1,e2)(e_1, e_2). It is nilpotent of index 2=s12 = s_1. On all of R3\mathbb{R}^3 the operator A−IA - I is not nilpotent, because (A−I)(2,1,1)=(2,1,1)(A - I)(2, 1, 1) = (2, 1, 1). Strictly upper triangular matrices are the standard examples. The one with rows (0,1,1)(0, 1, 1), (0,0,1)(0, 0, 1) and (0,0,0)(0, 0, 0) has a nonzero square and a zero cube, so its index is 33.

Dimension of a generalized eigenspace

Definition Suppose μT\mu_T splits over F\mathbb{F} as above, and let mjm_j be the algebraic multiplicity of λj\lambda_j, its multiplicity as a root of χT\chi_T. Then dim⁡Gλj(T)=mj\dim G_{\lambda_j}(T) = m_j and

χT(t)=∏j=1r(t−λj)mj.\chi_T(t) = \prod_{j=1}^{r} (t - \lambda_j)^{m_j}.

In particular χT\chi_T splits too, the dimensions m1+⋯+mrm_1 + \cdots + m_r add up to nn, and the geometric multiplicity dim⁡ker⁡(T−λjI)\dim \ker (T - \lambda_j I) is at most mjm_j, because the eigenspace lies inside Gλj(T)G_{\lambda_j}(T).

For the proof, join bases of the generalized eigenspaces into a basis of VV. The sum is direct and each piece is invariant, so the matrix of TT is block diagonal, with a block AjA_j of size dj=dim⁡Gλj(T)d_j = \dim G_{\lambda_j}(T) for each restriction, and χT=∏jχAj\chi_T = \prod_j \chi_{A_j}. Each block has the form Aj=λjI+MjA_j = \lambda_j I + M_j with MjM_j nilpotent. Every complex root of χMj\chi_{M_j} is an eigenvalue of MjM_j viewed as a complex matrix, which is still nilpotent, so that root is 00. A monic polynomial of degree djd_j whose only complex root is 00 equals tdjt^{d_j}, hence

χAj(t)=det⁡((t−λj)I−Mj)=χMj(t−λj)=(t−λj)dj.\begin{aligned} \chi_{A_j}(t) &= \det\bigl((t - \lambda_j) I - M_j\bigr) \\[0.4em] &= \chi_{M_j}(t - \lambda_j) = (t - \lambda_j)^{d_j}. \end{aligned}

Multiplying over jj and comparing the exponents of the distinct factors t−λjt - \lambda_j gives mj=djm_j = d_j. No Jordan basis is needed for this.

In plain words The algebraic multiplicity counts the whole generalized eigenspace, while the geometric multiplicity counts only its ordinary eigenvectors. The exponent in the minimal polynomial is a third number, the length of the longest chain.

Example For the matrix AA above, χA(t)=(t−1)2(t−2)\chi_A(t) = (t - 1)^2(t - 2). The eigenvalue 11 has algebraic multiplicity 2=dim⁡G1(A)2 = \dim G_1(A), geometric multiplicity 11, and exponent s1=2s_1 = 2 in μA\mu_A. The three numbers can all differ. On F4\mathbb{F}^4 let T=5I+E12T = 5I + E_{12}, where E12E_{12} has a single 11 in position (1,2)(1, 2). Then χT(t)=(t−5)4\chi_T(t) = (t - 5)^4 and G5(T)=F4G_5(T) = \mathbb{F}^4 has dimension 44. The eigenspace is the kernel of E12E_{12}, the hyperplane x2=0x_2 = 0 of dimension 33. Since E12≠0E_{12} \ne 0 and E122=0E_{12}^2 = 0, the minimal polynomial is (t−5)2(t - 5)^2.

Check yourself

  1. Let AA have rows (1,1,0)(1, 1, 0), (0,1,0)(0, 1, 0) and (0,0,2)(0, 0, 2). Find μA\mu_A, the projectors onto the generalized eigenspaces as polynomials in AA, and the split of x=(2,3,4)x = (2, 3, 4).
Show the answer

The characteristic polynomial is (t−1)2(t−2)(t - 1)^2(t - 2), and the minimal polynomial divides it and has both 11 and 22 as roots. The product (A−I)(A−2I)(A - I)(A - 2I) is not zero, because its first row is (0,−1,0)(0, -1, 0), so μA(t)=(t−1)2(t−2)\mu_A(t) = (t - 1)^2(t - 2). The Bézout relation (t−1)2−t(t−2)=1(t - 1)^2 - t(t - 2) = 1 for the coprime factors (t−1)2(t - 1)^2 and t−2t - 2 gives h2=(t−1)2h_2 = (t - 1)^2, so

P2=(A−I)2=diag⁡(0,0,1),P1=I−P2=diag⁡(1,1,0).\begin{aligned} P_2 &= (A - I)^2 = \operatorname{diag}(0, 0, 1), \\[0.4em] P_1 &= I - P_2 = \operatorname{diag}(1, 1, 0). \end{aligned}

The split is x=(2,3,0)+(0,0,4)x = (2, 3, 0) + (0, 0, 4). The first piece is killed by (A−I)2(A - I)^2 but not by A−IA - I, since (A−I)(2,3,0)=(3,0,0)(A - I)(2, 3, 0) = (3, 0, 0), so it is a generalized eigenvector of rank 22 for 11. The second piece is an eigenvector for 22.

  1. Let CC have rows (0,−1,0)(0, -1, 0), (1,0,0)(1, 0, 0) and (0,0,3)(0, 0, 3), acting on R3\mathbb{R}^3. Is R3\mathbb{R}^3 the direct sum of the real generalized eigenspaces of CC? Which decomposition survives over R\mathbb{R}, and what happens over C\mathbb{C}?
Show the answer

No. The characteristic polynomial is (t2+1)(t−3)(t^2 + 1)(t - 3). Neither C2+IC^2 + I nor C−3IC - 3I is zero, so the minimal polynomial is the whole product, which does not split over R\mathbb{R}. The only real eigenvalue is 33, and every kernel of a positive power of C−3IC - 3I is the line spanned by e3e_3, so the real generalized eigenspaces fill only one dimension. The primary decomposition still works with the coprime factors t2+1t^2 + 1 and t−3t - 3. From the Bézout relation

110(t2+1)−110(t+3)(t−3)=1,\begin{aligned} &\frac{1}{10}(t^2 + 1) \\[0.4em] &\quad - \frac{1}{10}(t + 3)(t - 3) = 1, \end{aligned}

the projector onto ker⁡(C−3I)\ker(C - 3I) is 110(C2+I)=diag⁡(0,0,1)\frac{1}{10}(C^2 + I) = \operatorname{diag}(0, 0, 1), and R3=ker⁡(C2+I)⊕ker⁡(C−3I)\mathbb{R}^3 = \ker(C^2 + I) \oplus \ker(C - 3I), the xyxy-plane plus the zz-axis. The plane is invariant but contains no real eigenvector. Over C\mathbb{C} the factor t2+1t^2 + 1 splits as (t−i)(t+i)(t - i)(t + i), and C3\mathbb{C}^3 is the direct sum of three generalized eigenspaces of dimension 11, spanned by (1,−i,0)(1, -i, 0), (1,i,0)(1, i, 0) and e3e_3.

  1. Let LL have rows (2,0,0)(2, 0, 0), (1,2,0)(1, 2, 0) and (1,1,3)(1, 1, 3). Find μL\mu_L, describe G2(L)G_2(L) and G3(L)G_3(L), and split x=(1,0,0)x = (1, 0, 0).
Show the answer

The matrix is lower triangular, so χL(t)=(t−2)2(t−3)\chi_L(t) = (t - 2)^2(t - 3). The matrix L−2IL - 2I has rows (0,0,0)(0, 0, 0), (1,0,0)(1, 0, 0) and (1,1,1)(1, 1, 1), so it has rank 22 and the eigenvalue 22 has only a line of eigenvectors. The minimal polynomial divides χL\chi_L and has both 22 and 33 as roots, and the product (L−2I)(L−3I)(L - 2I)(L - 3I) is not zero, since its second row is (−1,0,0)(-1, 0, 0), so μL(t)=(t−2)2(t−3)\mu_L(t) = (t - 2)^2(t - 3). The Bézout relation (t−2)2−(t−1)(t−3)=1(t - 2)^2 - (t - 1)(t - 3) = 1 gives h3=(t−2)2h_3 = (t - 2)^2, so

P3=(L−2I)2=[000000211],P2=I−P3=[100010−2−10].\begin{aligned} P_3 &= (L - 2I)^2 = \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 2 & 1 & 1 \end{bmatrix}, \\[0.4em] P_2 &= I - P_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & -1 & 0 \end{bmatrix}. \end{aligned}

So G2(L)=ker⁡(L−2I)2G_2(L) = \ker (L - 2I)^2 is the plane 2x1+x2+x3=02x_1 + x_2 + x_3 = 0, spanned by (1,0,−2)(1, 0, -2) and (0,1,−1)(0, 1, -1), and G3(L)G_3(L) is the line spanned by e3e_3, since Le3=3e3Le_3 = 3e_3. The dimensions 22 and 11 match the algebraic multiplicities. The split is

(1,0,0)=(1,0,−2)+(0,0,2).(1, 0, 0) = (1, 0, -2) + (0, 0, 2).

The first piece has rank 22, because (L−2I)(1,0,−2)=(0,1,−1)(L - 2I)(1, 0, -2) = (0, 1, -1), which is an eigenvector for 22.

How this lesson connects