Unit 8, lesson 42

Generalized Eigenvectors

Generalized eigenvectors and Jordan chains for operators with too few eigenvectors, why every chain is independent, why the nested kernels of powers stabilize, and the generalized eigenspace.

  • Video 42
  • Chains and Generalized Eigenspaces

The matrix with rows (2,1)(2, 1) and (0,2)(0, 2), a shear scaled by 22, has the single eigenvalue 22 but only one line of eigenvectors, so eigenvectors alone cannot describe how it acts on the plane. Subtracting 2I2I leaves an operator that sends e2e_2 to e1e_1 and e1e_1 to 00, so e2e_2 reaches an eigenvector after one step and vanishes after two. This lesson names such vectors, organizes them into chains, proves that every chain is independent, and shows that the kernels of the powers of T−λIT - \lambda I grow only until they stabilize, which defines the generalized eigenspace. Throughout, F\mathbb{F} is R\mathbb{R} or C\mathbb{C}, VV is a vector space over F\mathbb{F} of finite dimension nn, T ⁣:V→VT \colon V \to V is linear, λ∈F\lambda \in \mathbb{F} and N=T−λIN = T - \lambda I. A square matrix AA stands for the operator x↦Axx \mapsto Ax, and a tuple such as (2,3)(2, 3) stands for a column vector.

Key terms

Generalized eigenvector

Definition A generalized eigenvector of TT for λ\lambda is a nonzero vector v∈Vv \in V with Nkv=0N^k v = 0 for some positive integer kk. Its rank is the smallest such kk, so vv has rank kk when

Nkv=0andNk−1v≠0.N^k v = 0 \quad \text{and} \quad N^{k-1} v \ne 0.

The vectors of rank 11 are exactly the eigenvectors for λ\lambda. A vector of rank k≥2k \ge 2 is not an eigenvector for λ\lambda, because an eigenvector for λ\lambda is already killed by NN. If vv has rank kk, then Nk−1vN^{k-1} v is nonzero and killed by NN, so it is an eigenvector for λ\lambda. In particular, generalized eigenvectors for λ\lambda exist only when λ\lambda is an eigenvalue of TT.

In plain words An eigenvector is killed by one application of T−λIT - \lambda I. A generalized eigenvector may need several, and its rank counts how many. The last nonzero vector before zero is an ordinary eigenvector. This rank has nothing to do with the rank of a matrix.

Example For the matrix

A=[2102]A = \begin{bmatrix} 2 & 1 \\ 0 & 2 \end{bmatrix}

and λ=2\lambda = 2, the matrix N=A−2IN = A - 2I sends (x,y)(x, y) to (y,0)(y, 0), and N2=0N^2 = 0. Every vector (x,y)(x, y) with y≠0y \ne 0 has rank 22, for instance e2e_2, with Ne2=e1≠0Ne_2 = e_1 \ne 0 and N2e2=0N^2 e_2 = 0. It is not an eigenvector, since Ae2=(1,2)Ae_2 = (1, 2) is not a multiple of e2e_2. The vectors of rank 11 are the nonzero multiples of e1e_1.

Jordan chain

Definition Let vv be a generalized eigenvector of rank kk for λ\lambda. The Jordan chain generated by vv is the list v1,…,vkv_1, \dots, v_k with

vk=v,vk−1=Nv,…,v1=Nk−1v.\begin{gathered} v_k = v, \quad v_{k-1} = Nv, \quad \dots, \\[0.4em] v_1 = N^{k-1} v. \end{gathered}

It satisfies the chain equations v1≠0v_1 \ne 0, Nv1=0Nv_1 = 0 and Nvj=vj−1Nv_j = v_{j-1} for 2≤j≤k2 \le j \le k, and conversely any list with these properties is the chain generated by its top vector vkv_k. In terms of TT itself, the equations read

Tv1=λv1,Tvj=λvj+vj−1(2≤j≤k).\begin{aligned} Tv_1 &= \lambda v_1, \\[0.4em] Tv_j &= \lambda v_j + v_{j-1} \quad (2 \le j \le k). \end{aligned}

So the span of the chain is invariant under TT. It is the cyclic subspace generated by vkv_k, under NN and under TT alike, because polynomials in TT and polynomials in NN are the same operators. The chain is independent, as the next term shows, and in the ordered basis (v1,…,vk)(v_1, \dots, v_k) of its span the matrix of the restriction of TT is the k×kk \times k Jordan block

Jk(λ)=[λ1λ⋱⋱1λ],J_k(\lambda) = \begin{bmatrix} \lambda & 1 & & \\ & \lambda & \ddots & \\ & & \ddots & 1 \\ & & & \lambda \end{bmatrix},

with λ\lambda on the diagonal, ones directly above it and zeros elsewhere. This uses the bottom-first order. Listing the chain from the top down moves the ones below the diagonal.

In plain words The chain arrows describe NN, not TT. Each application of NN moves one step down the chain until the bottom vector, an eigenvector, is sent to zero. The operator TT adds the scalar part λ\lambda on top of that motion.

Example Let

A=[31−11].A = \begin{bmatrix} 3 & 1 \\ -1 & 1 \end{bmatrix}.

Its characteristic polynomial is t2−4t+4=(t−2)2t^2 - 4t + 4 = (t - 2)^2, and N=A−2IN = A - 2I has rows (1,1)(1, 1) and (−1,−1)(-1, -1), with N≠0N \ne 0 and N2=0N^2 = 0. Take v2=e1v_2 = e_1. Then v1=Ne1=(1,−1)v_1 = Ne_1 = (1, -1) and Nv1=0Nv_1 = 0. With the chain as the columns of PP,

P=[11−10],P−1AP=[2102].\begin{gathered} P = \begin{bmatrix} 1 & 1 \\ -1 & 0 \end{bmatrix}, \\[0.4em] P^{-1} A P = \begin{bmatrix} 2 & 1 \\ 0 & 2 \end{bmatrix}. \end{gathered}

The second column says Av2=2v2+v1Av_2 = 2v_2 + v_1, which checks directly, since Ae1=(3,−1)=2(1,0)+(1,−1)Ae_1 = (3, -1) = 2(1, 0) + (1, -1). In the reversed order (v2,v1)(v_2, v_1) the matrix has rows (2,0)(2, 0) and (1,2)(1, 2).

Independence of a chain

Definition Every Jordan chain v1,…,vkv_1, \dots, v_k for λ\lambda is linearly independent. To prove it, suppose

c1v1+c2v2+⋯+ckvk=0.c_1 v_1 + c_2 v_2 + \cdots + c_k v_k = 0.

The vector vjv_j becomes zero after jj applications of NN, so Nk−1N^{k-1} kills v1,…,vk−1v_1, \dots, v_{k-1} and sends vkv_k to v1v_1. The relation becomes ckv1=0c_k v_1 = 0, and v1≠0v_1 \ne 0 gives ck=0c_k = 0. Applying Nk−2N^{k-2} to the remaining relation leaves ck−1v1=0c_{k-1} v_1 = 0, so ck−1=0c_{k-1} = 0, and continuing down to N0=IN^0 = I gives cj=0c_j = 0 for every jj. Consequently a generalized eigenvector has rank at most n=dim⁡Vn = \dim V, since a vector of rank kk generates a chain of kk independent vectors in VV.

In plain words Apply NN exactly often enough to wipe out every vector except the top one. What is left is a multiple of the nonzero bottom vector, so the top coefficient is zero. Then repeat one level lower. The argument needs the bottom vector to be nonzero, which is part of being a chain.

Example Let

B=[111011001]B = \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix}

and N=B−IN = B - I. Then N2N^2 has a single 11 in position (1,3)(1, 3) and N3=0N^3 = 0, so e3e_3 has rank 33. Its chain is v3=e3v_3 = e_3, v2=Ne3=(1,1,0)v_2 = Ne_3 = (1, 1, 0) and v1=Nv2=(1,0,0)v_1 = Nv_2 = (1, 0, 0). For a relation c1v1+c2v2+c3v3=0c_1 v_1 + c_2 v_2 + c_3 v_3 = 0, applying N2N^2 gives c3v1=0c_3 v_1 = 0. With c3=0c_3 = 0, applying NN gives c2v1=0c_2 v_1 = 0, and then c1v1=0c_1 v_1 = 0. As a cross-check, the matrix with columns v1,v2,v3v_1, v_2, v_3 is upper triangular with ones on the diagonal, so its determinant is 11.

Nested kernels

Definition The kernels of the powers of NN form an increasing sequence of subspaces

{0}=ker⁡N0⊆ker⁡N⊆ker⁡N2⊆⋯ ,\begin{aligned} \{0\} &= \ker N^0 \subseteq \ker N \\[0.4em] &\subseteq \ker N^2 \subseteq \cdots, \end{aligned}

because Nkv=0N^k v = 0 implies Nk+1v=0N^{k+1} v = 0. Each ker⁡Nk\ker N^k is invariant under TT, because TT commutes with NkN^k, so Nkv=0N^k v = 0 gives Nk(Tv)=TNkv=0N^k(Tv) = T N^k v = 0. The kernel of NN itself is the eigenspace Eλ=ker⁡(T−λI)E_\lambda = \ker(T - \lambda I). The generalized eigenvectors of rank kk are exactly the vectors in ker⁡Nk∖ker⁡Nk−1\ker N^k \setminus \ker N^{k-1}, and the difference

dim⁡ker⁡Nk−dim⁡ker⁡Nk−1\dim \ker N^k - \dim \ker N^{k-1}

measures how many independent directions are first killed at step kk. It concerns only the scalar λ\lambda fixed in NN and says nothing about how many distinct eigenvalues TT has.

In plain words Sort vectors by how many applications of NN they need before they vanish. The kkth kernel collects the vectors that vanish within kk steps, and these kernels can only grow as kk increases.

Example Let

A=[210021002]A = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 2 & 1 \\ 0 & 0 & 2 \end{bmatrix}

and N=A−2IN = A - 2I. Then N(x,y,z)=(y,z,0)N(x, y, z) = (y, z, 0), N2(x,y,z)=(z,0,0)N^2(x, y, z) = (z, 0, 0) and N3=0N^3 = 0. So ker⁡N\ker N is the xx-axis, ker⁡N2\ker N^2 is the xyxy-plane and ker⁡N3=R3\ker N^3 = \mathbb{R}^3, with dimensions 1,2,31, 2, 3 and one new direction at each step. The chain e3→e2→e1→0e_3 \to e_2 \to e_1 \to 0 under NN explains the coupling seen when AA itself is applied. Since II and NN commute, the binomial formula gives A2=(2I+N)2=4I+4N+N2A^2 = (2I + N)^2 = 4I + 4N + N^2, so

A2e3=4e3+4e2+e1=(1,4,4).A^2 e_3 = 4e_3 + 4e_2 + e_1 = (1, 4, 4).

The eigenvalue alone predicts only the term 4e34e_3. The other two terms come from the chain.

Stabilization of kernels

Definition If ker⁡Nm=ker⁡Nm+1\ker N^m = \ker N^{m+1} for some m≥0m \ge 0, then ker⁡Nm+j=ker⁡Nm\ker N^{m+j} = \ker N^m for every j≥0j \ge 0. To prove it, let v∈ker⁡Nm+2v \in \ker N^{m+2}. Then Nv∈ker⁡Nm+1=ker⁡NmNv \in \ker N^{m+1} = \ker N^m, so Nm+1v=0N^{m+1} v = 0 and v∈ker⁡Nm+1v \in \ker N^{m+1}. Hence ker⁡Nm+2=ker⁡Nm+1\ker N^{m+2} = \ker N^{m+1}, and induction gives the rest. In dimension nn the kernels stabilize by step nn. Until the first equality each step raises the dimension by at least 11, and no dimension exceeds nn, so ker⁡Nm=ker⁡Nm+1\ker N^m = \ker N^{m+1} for some m≤nm \le n. Therefore

ker⁡Nn=ker⁡Nn+1=ker⁡Nn+2=⋯ ,\begin{aligned} \ker N^n &= \ker N^{n+1} \\[0.4em] &= \ker N^{n+2} = \cdots, \end{aligned}

and ker⁡Nn\ker N^n contains every vector that some power of NN kills.

In plain words Once a step adds nothing new, no later step can add anything either. Every productive step adds at least one dimension, so the growth stops after at most nn steps.

Example Let B=diag⁡(2,2,3)B = \operatorname{diag}(2, 2, 3) and λ=2\lambda = 2. Then N=B−2I=diag⁡(0,0,1)N = B - 2I = \operatorname{diag}(0, 0, 1) and N2=diag⁡(0,0,1)N^2 = \operatorname{diag}(0, 0, 1), so

ker⁡N=ker⁡N2=span⁡(e1,e2),\ker N = \ker N^2 = \operatorname{span}(e_1, e_2),

and the kernels have stabilized at m=1m = 1. There is no generalized eigenvector of rank 22 for 22. The eigenvalue 22 is repeated, but it has two independent ordinary eigenvectors instead of a longer chain. For the matrix AA with rows (2,1,0)(2, 1, 0), (0,2,1)(0, 2, 1) and (0,0,2)(0, 0, 2), the kernel dimensions 1,2,31, 2, 3 stabilize only at step 3=n3 = n.

Generalized eigenspace

Definition The generalized eigenspace of TT for λ\lambda is

Gλ(T)=ker⁡(T−λI)n,n=dim⁡V.\begin{gathered} G_\lambda(T) = \ker (T - \lambda I)^n, \\[0.4em] n = \dim V. \end{gathered}

By stabilization it consists of the zero vector together with all generalized eigenvectors for λ\lambda, that is, of all vv with (T−λI)kv=0(T - \lambda I)^k v = 0 for some k≥1k \ge 1. It is a TT-invariant subspace that contains the eigenspace EλE_\lambda, and it is nonzero exactly when λ\lambda is an eigenvalue of TT. On Gλ(T)G_\lambda(T) the power NnN^n is zero, but NN need not be nilpotent on all of VV, because vectors belonging to other eigenvalues can survive every power of NN.

In plain words The generalized eigenspace collects every vector that repeated removal of λ\lambda eventually sends to zero. The exponent nn is always large enough, because by then nothing new can enter.

Example Let

C=[210020005]C = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 5 \end{bmatrix}

and λ=2\lambda = 2. For N=C−2IN = C - 2I and every k≥2k \ge 2, the power NkN^k has the single nonzero entry 3k3^k in position (3,3)(3, 3). The kernel dimensions are 1,2,21, 2, 2, so G2(C)=ker⁡N3=span⁡(e1,e2)G_2(C) = \ker N^3 = \operatorname{span}(e_1, e_2), while the eigenspace E2E_2 is only the line spanned by e1e_1. The vector e3e_3 is never killed, since Nke3=3ke3N^k e_3 = 3^k e_3, so NN is not nilpotent on R3\mathbb{R}^3. That vector lies in the generalized eigenspace for the other eigenvalue 55.

Check yourself

  1. Let AA have rows (5,1)(5, 1) and (0,5)(0, 5). Show that v=(2,3)v = (2, 3) is a generalized eigenvector for 55, find its rank and its Jordan chain, and check the chain equations for AA.
Show the answer

The matrix N=A−5IN = A - 5I sends (x,y)(x, y) to (y,0)(y, 0). So Nv=(3,0)≠0Nv = (3, 0) \ne 0 and N2v=N(3,0)=0N^2 v = N(3, 0) = 0, and vv has rank 22. Its chain is v2=(2,3)v_2 = (2, 3) and v1=(3,0)v_1 = (3, 0). For AA itself,

Av2=(13,15)=5(2,3)+(3,0)=5v2+v1,\begin{aligned} Av_2 &= (13, 15) \\[0.4em] &= 5(2, 3) + (3, 0) \\[0.4em] &= 5v_2 + v_1, \end{aligned}

and Av1=(15,0)=5v1Av_1 = (15, 0) = 5v_1, so the bottom of the chain is an eigenvector.

  1. Let MM have rows (2,0,1)(2, 0, 1), (0,2,0)(0, 2, 0) and (0,0,2)(0, 0, 2), so that χM(t)=(t−2)3\chi_M(t) = (t - 2)^3. Find dim⁡ker⁡(M−2I)k\dim \ker (M - 2I)^k for k=1,2,3k = 1, 2, 3 and the largest rank of a generalized eigenvector for 22. Is there a chain of length 33?
Show the answer

The matrix N=M−2IN = M - 2I has a single 11 in position (1,3)(1, 3), so N(x,y,z)=(z,0,0)N(x, y, z) = (z, 0, 0) and N2=0N^2 = 0. Hence ker⁡N\ker N is the plane z=0z = 0, of dimension 22, and ker⁡N2=ker⁡N3=R3\ker N^2 = \ker N^3 = \mathbb{R}^3, so the dimensions are 2,3,32, 3, 3. The largest rank is 22. For example e3e_3 has the chain e3→e1→0e_3 \to e_1 \to 0, and e2e_2 is an eigenvector outside that chain. There is no chain of length 33, because N2=0N^2 = 0 already kills every vector. A triple root of the characteristic polynomial does not force a chain of length 33.

  1. Let λ≠ν\lambda \ne \nu be scalars in F\mathbb{F}. Prove that no nonzero vector is a generalized eigenvector of TT for both λ\lambda and ν\nu.
Show the answer

Suppose vv has rank kk for λ\lambda and (T−νI)mv=0(T - \nu I)^m v = 0 for some m≥1m \ge 1. Let w=(T−λI)k−1vw = (T - \lambda I)^{k-1} v, which is an eigenvector for λ\lambda. Polynomials in TT commute, so

(T−νI)mw=(T−λI)k−1(T−νI)mv=0.\begin{aligned} (T - \nu I)^m w &= (T - \lambda I)^{k-1} (T - \nu I)^m v \\[0.4em] &= 0. \end{aligned}

On the other hand Tw=λwTw = \lambda w gives (T−νI)w=(λ−ν)w(T - \nu I)w = (\lambda - \nu) w, hence (T−νI)mw=(λ−ν)mw(T - \nu I)^m w = (\lambda - \nu)^m w. This is not zero, because λ≠ν\lambda \ne \nu and w≠0w \ne 0. The contradiction shows that Gλ(T)∩Gν(T)={0}G_\lambda(T) \cap G_\nu(T) = \{0\}.

How this lesson connects