Repeated Eigenvalues and Generalized Eigenvectors, differential equations lesson

With $t$ as the independent variable and a prime meaning $d/dt$ , take the system $X' = AX$ whose coefficient matrix $A$ has rows $(2, 1)$ and $(0, 2)$ . The eigenvalue $2$ is a double root of the characteristic polynomial, yet every eigenvector lies on one line, so the eigenvector method gives only the solution $e^{2t}(1, 0)^{T}$ and its multiples. A system of two first-order equations needs two independent solutions. This lesson shows how to tell a repeated eigenvalue with a full set of eigenvectors from one with a missing direction, and how a generalized eigenvector together with a factor of $t$ supplies the missing solution.

Key terms

Algebraic and geometric multiplicity

Definition Let $A$ be a constant $n \times n$ matrix with real or complex entries, and allow eigenvalues and eigenvectors to be complex, with eigenvectors in $\mathbb{C}^n$ . The characteristic polynomial $p(\lambda) = \det(A - \lambda I)$ factors as

p(\lambda) = \prod_{j=1}^{r} (\lambda_j - \lambda)^{m_j},

where $\lambda_1, \dots, \lambda_r$ are the distinct eigenvalues and $m_1 + \cdots + m_r = n$ . The algebraic multiplicity of $\lambda_j$ is the exponent $m_j$ . The geometric multiplicity of $\lambda_j$ is the number of linearly independent eigenvectors for $\lambda_j$ , which is the dimension of its eigenspace,

\dim \ker(A - \lambda_j I).

For every eigenvalue the geometric multiplicity is at least $1$ and at most the algebraic multiplicity.

In plain words The algebraic multiplicity counts how often an eigenvalue appears as a root. The geometric multiplicity counts how many independent vectors the matrix simply multiplies by that eigenvalue. The second count can fall short of the first, but it can never exceed it.

Example The matrices $2I$ and $A$ both have characteristic polynomial $(2 - \lambda)^2$ , so for each of them the eigenvalue $2$ has algebraic multiplicity $2$ . For $2I$ every nonzero vector is an eigenvector, and the geometric multiplicity is $2$ . For $A$ ,

A - 2I = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix},

and $(A - 2I)x = 0$ for $x = (x_1, x_2)^{T}$ says only that $x_2 = 0$ . Every eigenvector is a nonzero multiple of $v = (1, 0)^{T}$ , so the geometric multiplicity is $1$ .

Diagonalizable matrix

Definition An $n \times n$ matrix $A$ is diagonalizable if $A = VDV^{-1}$ for an invertible matrix $V$ and a diagonal matrix $D$ , where complex entries are allowed. The columns $v_1, \dots, v_n$ of $V$ then form an eigenbasis of $\mathbb{C}^n$ , with $Av_j = \lambda_j v_j$ , where $\lambda_j$ is the $j$ th diagonal entry of $D$ . Each eigenvalue appears on that diagonal as often as its algebraic multiplicity. A matrix is diagonalizable exactly when every eigenvalue has equal algebraic and geometric multiplicities. Having $n$ distinct eigenvalues is enough for this, but it is not necessary. For a diagonalizable $A$ , every solution of $X' = AX$ is

\begin{aligned} X(t) = {}&c_1e^{\lambda_1 t}v_1 \\[0.4em] &+ \cdots + c_ne^{\lambda_n t}v_n \end{aligned}

for exactly one choice of the constants $c_1, \dots, c_n$ .

In plain words A repeated eigenvalue causes no trouble as long as it comes with as many independent eigenvectors as its multiplicity. Each eigenvector then gives its own exponential mode, and together the modes describe every solution.

Example The matrix $2I$ has the repeated eigenvalue $2$ , and any two independent vectors form an eigenbasis. The solution with $X(0) = X_0$ is $X(t) = e^{2t}X_0$ , which only rescales the initial state. For a $2 \times 2$ matrix this is the only way a repeated eigenvalue can come with a full eigenbasis. If the eigenspace of $\lambda$ is all of $\mathbb{C}^2$ , then $(A - \lambda I)x = 0$ for every $x$ , and so $A = \lambda I$ .

Defective matrix

Definition A square matrix is defective if it is not diagonalizable. This happens exactly when some eigenvalue has a geometric multiplicity smaller than its algebraic multiplicity. Only a repeated eigenvalue can be responsible, since algebraic multiplicity $1$ forces geometric multiplicity $1$ . For a defective $n \times n$ matrix the eigenvector solutions $e^{\lambda t}v$ supply only as many independent solutions of $X' = AX$ as the sum of the geometric multiplicities, which is less than $n$ .

In plain words A defective matrix is missing eigenvector directions. Multiplying the eigenvector solution by another constant only rescales it, so the missing solution has to come from a different kind of vector.

Example The matrix $A$ with rows $(2, 1)$ and $(0, 2)$ is defective, because its eigenvalue $2$ has algebraic multiplicity $2$ and geometric multiplicity $1$ . The eigenvector method gives $e^{2t}v$ with $v = (1, 0)^{T}$ , and every multiple $ce^{2t}v$ is the same mode with another amplitude. The guess $te^{2t}v$ , modeled on the scalar case, fails. Its derivative is $e^{2t}v + 2te^{2t}v$ , while

A\left(te^{2t}v\right) = 2te^{2t}v,

so a term $e^{2t}v$ is left over.

Generalized eigenvector

Definition Let $\lambda$ be an eigenvalue of $A$ . A nonzero vector $w$ is a generalized eigenvector of $A$ for $\lambda$ if

(A - \lambda I)^k w = 0

for some positive integer $k$ . The smallest such $k$ is the rank of $w$ , and the generalized eigenvectors of rank $1$ are exactly the ordinary eigenvectors. If $\lambda$ has algebraic multiplicity $m$ , the generalized eigenvectors for $\lambda$ together with the zero vector form the generalized eigenspace $\ker(A - \lambda I)^m$ , and its dimension is $m$ .

In plain words One application of $A - \lambda I$ sends an eigenvector to zero. A generalized eigenvector may need several applications, and each one moves it a step closer to an ordinary eigenvector. The generalized eigenvectors of an eigenvalue fill a space whose dimension is the algebraic multiplicity, so the missing directions are always found there.

Example For the matrix $A$ above, take $w = (0, 1)^{T}$ . Then $(A - 2I)w = (1, 0)^{T} = v$ , which is not zero, and $(A - 2I)^2w = (A - 2I)v = 0$ . So $w$ is a generalized eigenvector of rank $2$ . The choice is not unique. Since $(A - 2I)^2 = 0$ , every vector off the eigenvector line has rank $2$ . For example $(5, 1)^{T}$ also works, and $A - 2I$ sends it to $v$ as well.

Jordan chain

Definition A Jordan chain of length $k$ for the eigenvalue $\lambda$ is a list of vectors $v_1, \dots, v_k$ with $v_1 \ne 0$ and

\begin{gathered} (A - \lambda I)v_1 = 0, \\[0.4em] (A - \lambda I)v_j = v_{j-1} \end{gathered}

for $j = 2, \dots, k$ . The vectors of a chain are linearly independent, $v_1$ is an eigenvector, and $v_j$ is a generalized eigenvector of rank $j$ . Each chain gives $k$ linearly independent solutions of $X' = AX$ , namely

X_j(t) = e^{\lambda t}\sum_{i=1}^{j} \frac{t^{j-i}}{(j-i)!}\,v_i

for $j = 1, \dots, k$ . The first two are $X_1 = e^{\lambda t}v_1$ and $X_2 = e^{\lambda t}(tv_1 + v_2)$ . By the Jordan form theorem, $\mathbb{C}^n$ has a basis made of Jordan chains of $A$ , and the solutions from all the chains together form a fundamental set. For a $2 \times 2$ matrix with a repeated eigenvalue $\lambda$ and $A \ne \lambda I$ , the Cayley-Hamilton theorem gives $(A - \lambda I)^2 = 0$ . So any vector $w$ outside the eigenspace starts a chain of length $2$ , with $v = (A - \lambda I)w$ as its eigenvector.

In plain words The matrix $A - \lambda I$ walks down the chain from $v_k$ to $v_{k-1}$ and so on to $v_1$ , and then to zero. Differentiating the factor $t$ in $X_2$ produces an extra $v_1$ , and $A$ produces the same extra $v_1$ from $v_2$ . That match is why the second solution carries a factor of $t$ .

Example For the matrix $A$ above, $v = (1, 0)^{T}$ and $w = (0, 1)^{T}$ form a chain of length $2$ . The general solution is

\begin{aligned} X(t) &= c_1e^{2t}v \\[0.4em] &\quad + c_2e^{2t}(tv + w) \\[0.4em] &= e^{2t}\begin{pmatrix} c_1 + c_2t \\ c_2 \end{pmatrix}. \end{aligned}

The initial state $X(0) = (0, 1)^{T}$ gives $c_1 = 0$ and $c_2 = 1$ , so $X(t) = e^{2t}(t, 1)^{T}$ . As a check, the first component $te^{2t}$ has derivative $e^{2t}(1 + 2t)$ , which equals twice the first component plus the second. The second component $e^{2t}$ has derivative twice itself, as the second row of $A$ requires.

Nilpotent part

Definition A square matrix $N$ is nilpotent if $N^k = 0$ for some positive integer $k$ . If $\lambda$ is the only eigenvalue of the $n \times n$ matrix $A$ , then $N = A - \lambda I$ is nilpotent, with $N^n = 0$ by the Cayley-Hamilton theorem, and $A = \lambda I + N$ splits $A$ into a multiple of the identity and its nilpotent part. The matrix exponential $e^{tA} = I + tA + \tfrac{t^2}{2!}A^2 + \cdots$ , studied in the next lesson, gives the solution $X(t) = e^{tA}X_0$ with $X(0) = X_0$ . Because $\lambda I$ commutes with $N$ , it splits as $e^{tA} = e^{\lambda t}e^{tN}$ , and the series for $e^{tN}$ stops,

\begin{aligned} e^{tA} &= e^{\lambda t}\Big(I + tN + \cdots \\[0.4em] &\qquad + \frac{t^{k-1}}{(k-1)!}N^{k-1}\Big), \end{aligned}

where $k$ is the smallest power with $N^k = 0$ . This $k$ equals the length of the longest Jordan chain for $\lambda$ . A matrix with several eigenvalues splits in the same way on each generalized eigenspace. In general $A = S + N$ with $S$ diagonalizable, $N$ nilpotent and $SN = NS$ , and these conditions determine $S$ and $N$ uniquely. This $N$ is called the nilpotent part of $A$ .

In plain words The identity part of $A$ only scales every state by $e^{\lambda t}$ , and the nilpotent part supplies polynomial factors in $t$ . Because some power of $N$ is zero, the polynomial stops, and its degree is one less than the length of the longest chain.

Example For the matrix $A$ above, $N = A - 2I$ has a single $1$ above the diagonal and $N^2 = 0$ , so

\begin{aligned} e^{tA} &= e^{2t}(I + tN) \\[0.4em] &= e^{2t}\begin{pmatrix} 1 & t \\ 0 & 1 \end{pmatrix}. \end{aligned}

Its columns are the two chain solutions $e^{2t}v$ and $e^{2t}(tv + w)$ . For the $3 \times 3$ matrix $J$ with $\lambda$ on the diagonal, $1$ directly above it and zeros elsewhere, $N = J - \lambda I$ has $N^2 \ne 0$ but $N^3 = 0$ . The chain has length $3$ , and a factor $t^2/2$ appears,

e^{tJ} = e^{\lambda t}\begin{pmatrix} 1 & t & \tfrac{t^2}{2} \\ 0 & 1 & t \\ 0 & 0 & 1 \end{pmatrix}.

Check yourself

For each matrix, find the algebraic and geometric multiplicity of its eigenvalue and decide whether it is defective.

\begin{gathered} P = \begin{pmatrix} -3 & 0 \\ 0 & -3 \end{pmatrix}, \\[0.4em] Q = \begin{pmatrix} -3 & 0 \\ 5 & -3 \end{pmatrix} \end{gathered}

Show the answer

Both matrices have characteristic polynomial $(-3 - \lambda)^2$ , so in each the eigenvalue $-3$ has algebraic multiplicity $2$ . For $P$ the matrix $P + 3I$ is zero, so every nonzero vector is an eigenvector. The geometric multiplicity is $2$ , and $P = -3I$ is diagonalizable. For $Q$ ,

Q + 3I = \begin{pmatrix} 0 & 0 \\ 5 & 0 \end{pmatrix},

and $(Q + 3I)x = 0$ means $5x_1 = 0$ . The eigenvectors are the nonzero multiples of $(0, 1)^{T}$ , so the geometric multiplicity is $1$ and $Q$ is defective. The repeated eigenvalue alone does not decide the question.

Solve $X' = AX$ with $X(0) = (1, 2)^{T}$ for the matrix

A = \begin{pmatrix} 1 & -1 \\ 1 & 3 \end{pmatrix}.

Show the answer

The characteristic polynomial is $(1 - \lambda)(3 - \lambda) + 1 = \lambda^2 - 4\lambda + 4 = (\lambda - 2)^2$ . Then

A - 2I = \begin{pmatrix} -1 & -1 \\ 1 & 1 \end{pmatrix},

whose kernel is spanned by $v = (1, -1)^{T}$ , so $A$ is defective. The equation $(A - 2I)w = v$ reduces to $-w_1 - w_2 = 1$ , and $w = (-1, 0)^{T}$ is one solution. The general solution is $c_1e^{2t}v + c_2e^{2t}(tv + w)$ . At $t = 0$ it equals $c_1v + c_2w = (c_1 - c_2, -c_1)^{T}$ , and setting this equal to $(1, 2)^{T}$ gives $c_1 = -2$ and $c_2 = -3$ . Hence

X(t) = e^{2t}\begin{pmatrix} 1 - 3t \\ 2 + 3t \end{pmatrix}.

To check, the first component has derivative $e^{2t}(-1 - 6t)$ , which is the first component minus the second. The second component has derivative $e^{2t}(7 + 6t)$ , which is the first component plus three times the second. The nilpotent part gives the same answer in one line. Since $(A - 2I)^2 = 0$ and $(A - 2I)X(0) = (-3, 3)^{T}$ ,

\begin{aligned} X(t) &= e^{2t}X(0) \\[0.4em] &\quad + te^{2t}(A - 2I)X(0) \\[0.4em] &= e^{2t}(1 - 3t, 2 + 3t)^{T}. \end{aligned}

The matrix $B$ with rows $(-1, 1)$ and $(0, -1)$ is defective. A classmate argues that the factor $t$ in its second chain solution makes the solutions of $X' = BX$ grow. Decide whether every solution tends to the origin, and compare with the nilpotent matrix $N$ with rows $(0, 1)$ and $(0, 0)$ .

Show the answer

The vectors $v = (1, 0)^{T}$ and $w = (0, 1)^{T}$ form a chain for $B$ with $\lambda = -1$ , since $(B + I)w = v$ and $(B + I)v = 0$ . Every solution is therefore

X(t) = e^{-t}\begin{pmatrix} c_1 + c_2t \\ c_2 \end{pmatrix}.

Because $te^{-t} \to 0$ as $t \to \infty$ , both components tend to $0$ for every choice of the constants, so every solution approaches the origin. The negative exponential rate outweighs the polynomial factor. For $N$ the eigenvalue is $0$ , the solutions are $(c_1 + c_2t, c_2)^{T}$ , and every solution with $c_2 \ne 0$ is unbounded. The factor $t$ decides the long-term behavior only when the real part of the eigenvalue is zero.

Repeated Eigenvalues and Generalized Eigenvectors

Key terms

Algebraic and geometric multiplicity

Diagonalizable matrix

Defective matrix

Generalized eigenvector

Jordan chain

Nilpotent part

Check yourself

How this lesson connects

Builds on

Used later in