Unit 6, lesson 50

Linear Systems and Eigenvalues

How to write coupled linear equations as one matrix equation, solve it with eigenvalues, eigenvectors and the matrix exponential, and read off whether solutions decay, grow or circle.

  • Theory part 99
  • Practice part 100
  • Systems of Equations

Two tanks of brine drain into each other, or a mass on a spring has both a position and a velocity. Each quantity changes at a rate that depends on both, so neither equation can be solved alone. Stacking the quantities into one vector turns the pair into a single equation x′=Ax\mathbf{x}' = A\mathbf{x}, and the eigenvalues of the matrix AA decide almost everything about how the solutions behave.

Key terms

First-order vector systems

Definition A first-order linear system for nn unknown functions x1(t),…,xn(t)x_1(t), \dots, x_n(t) is

x′(t)=A(t) x(t)+g(t),\mathbf{x}'(t) = A(t)\,\mathbf{x}(t) + \mathbf{g}(t),

where x=⟨x1,…,xn⟩\mathbf{x} = \langle x_1, \dots, x_n \rangle is written as a column, A(t)A(t) is an n×nn \times n matrix of functions and g(t)\mathbf{g}(t) is a vector of functions. It is homogeneous when g=0\mathbf{g} = \mathbf{0} and has constant coefficients when AA does not depend on tt. If the entries of AA and g\mathbf{g} are continuous on an open interval II containing t0t_0, then for every vector x0\mathbf{x}_0 the initial-value problem with x(t0)=x0\mathbf{x}(t_0) = \mathbf{x}_0 has exactly one solution, and it exists on all of II.

In plain words List all the changing quantities in one vector. Each row of the system says how fast one of them changes, in terms of all of them.

Example The equation y′′+3y′+2y=0y'' + 3y' + 2y = 0 becomes a system once you name the state components first. Set x1=yx_1 = y and x2=y′x_2 = y'. Then x1′=x2x_1' = x_2 and x2′=y′′=−2x1−3x2x_2' = y'' = -2x_1 - 3x_2, so

x′=[01−2−3]x.\mathbf{x}' = \begin{bmatrix} 0 & 1 \\ -2 & -3 \end{bmatrix} \mathbf{x}.

Matrices and the solution space

Definition For an n×nn \times n real matrix AA and a vector x\mathbf{x} in Rn\mathbb{R}^n, the product AxA\mathbf{x} is the vector whose iith entry is ∑jaijxj\sum_{j} a_{ij} x_j. The map x↦Ax\mathbf{x} \mapsto A\mathbf{x} is linear. For the homogeneous system x′=Ax\mathbf{x}' = A\mathbf{x} with AA constant, every linear combination of solutions is again a solution, and the solutions form a vector space of dimension nn. Solutions x1,…,xn\mathbf{x}_1, \dots, \mathbf{x}_n form a fundamental set when their Wronskian

W(t)=det⁡[ x1(t)  ⋯  xn(t) ]W(t) = \det\bigl[\,\mathbf{x}_1(t) \;\cdots\; \mathbf{x}_n(t)\,\bigr]

is nonzero. For solutions of the same system on an interval, WW is either zero everywhere or zero nowhere, and for a fundamental set every solution equals c1x1+⋯+cnxnc_1 \mathbf{x}_1 + \dots + c_n \mathbf{x}_n for unique constants.

In plain words The matrix is the rule that turns the current state into its velocity. Because that rule is linear, you only need nn independent solutions, and every other solution is a mix of them.

Example For the matrix AA above and x=⟨1,−1⟩\mathbf{x} = \langle 1, -1 \rangle,

Ax=[0⋅1+1⋅(−1)−2⋅1−3⋅(−1)]=[−11]=−x.\begin{aligned} A\mathbf{x} &= \begin{bmatrix} 0 \cdot 1 + 1 \cdot (-1) \\ -2 \cdot 1 - 3 \cdot (-1) \end{bmatrix} \\[0.4em] &= \begin{bmatrix} -1 \\ 1 \end{bmatrix} = -\mathbf{x}. \end{aligned}

The matrix sends this vector to −1-1 times itself, which is the idea of the next section.

Eigenvalues and eigenvectors

Definition A number λ\lambda, real or complex, is an eigenvalue of the n×nn \times n matrix AA if Av=λvA\mathbf{v} = \lambda\mathbf{v} for some nonzero vector v\mathbf{v}, called an eigenvector for λ\lambda. The eigenvalues are the roots of the characteristic equation det⁡(A−λI)=0\det(A - \lambda I) = 0. For a 2×22 \times 2 matrix with trace TT and determinant DD this equation is

λ2−Tλ+D=0.\lambda^2 - T\lambda + D = 0.

If Av=λvA\mathbf{v} = \lambda\mathbf{v}, then x(t)=eλtv\mathbf{x}(t) = e^{\lambda t}\mathbf{v} solves x′=Ax\mathbf{x}' = A\mathbf{x}, since both sides equal λeλtv\lambda e^{\lambda t}\mathbf{v}.

In plain words An eigenvector is a direction the matrix only stretches. A solution that starts along it stays on that line and grows or shrinks by the factor eλte^{\lambda t}.

Example For A=[01−2−3]A = \begin{bmatrix} 0 & 1 \\ -2 & -3 \end{bmatrix} the trace is −3-3 and the determinant is 22, so λ2+3λ+2=0\lambda^2 + 3\lambda + 2 = 0 and λ=−1\lambda = -1 or λ=−2\lambda = -2. Solving (A+I)v=0(A + I)\mathbf{v} = \mathbf{0} gives v1=⟨1,−1⟩\mathbf{v}_1 = \langle 1, -1 \rangle, and solving (A+2I)v=0(A + 2I)\mathbf{v} = \mathbf{0} gives v2=⟨1,−2⟩\mathbf{v}_2 = \langle 1, -2 \rangle.

Diagonalizable systems

Definition The matrix AA is diagonalizable if it has nn linearly independent eigenvectors v1,…,vn\mathbf{v}_1, \dots, \mathbf{v}_n, with eigenvalues λ1,…,λn\lambda_1, \dots, \lambda_n. This holds in particular when the nn eigenvalues are distinct. Then A=PΛP−1A = P\Lambda P^{-1} with P=[ v1⋯vn ]P = [\,\mathbf{v}_1 \cdots \mathbf{v}_n\,] and Λ\Lambda diagonal, and the general solution of x′=Ax\mathbf{x}' = A\mathbf{x} is

x(t)=c1eλ1tv1+⋯+cneλntvn.\mathbf{x}(t) = c_1 e^{\lambda_1 t}\mathbf{v}_1 + \dots + c_n e^{\lambda_n t}\mathbf{v}_n.

When AA is real and λ=α+iβ\lambda = \alpha + i\beta with β≠0\beta \ne 0 has eigenvector v\mathbf{v}, the real and imaginary parts of eλtve^{\lambda t}\mathbf{v} are two linearly independent real solutions, and they replace the pair for λ\lambda and λˉ\bar\lambda.

In plain words Split the initial state into eigenvector pieces. Each piece evolves on its own with its own exponential, and you add them back up.

Example Solve x′=Ax\mathbf{x}' = A\mathbf{x} with the matrix above and x(0)=⟨1,0⟩\mathbf{x}(0) = \langle 1, 0 \rangle. The conditions c1+c2=1c_1 + c_2 = 1 and −c1−2c2=0-c_1 - 2c_2 = 0 give c1=2c_1 = 2 and c2=−1c_2 = -1, so

x(t)=2e−t[1−1]−e−2t[1−2].\mathbf{x}(t) = 2e^{-t}\begin{bmatrix} 1 \\ -1 \end{bmatrix} - e^{-2t}\begin{bmatrix} 1 \\ -2 \end{bmatrix}.

The first component y=2e−t−e−2ty = 2e^{-t} - e^{-2t} solves y′′+3y′+2y=0y'' + 3y' + 2y = 0 with y(0)=1y(0) = 1 and y′(0)=0y'(0) = 0. For an oscillatory case take B=[−12−2−1]B = \begin{bmatrix} -1 & 2 \\ -2 & -1 \end{bmatrix}, with λ=−1+2i\lambda = -1 + 2i and v=⟨1,i⟩\mathbf{v} = \langle 1, i \rangle. Taking real and imaginary parts of e(−1+2i)tve^{(-1 + 2i)t}\mathbf{v} gives

e−t[cos⁡2t−sin⁡2t],e−t[sin⁡2tcos⁡2t].e^{-t}\begin{bmatrix} \cos 2t \\ -\sin 2t \end{bmatrix}, \qquad e^{-t}\begin{bmatrix} \sin 2t \\ \cos 2t \end{bmatrix}.

Repeated eigenvalues

Definition Suppose the real 2×22 \times 2 matrix AA has a double eigenvalue λ\lambda, which is then real. There are two cases. If A=λIA = \lambda I, every nonzero vector is an eigenvector and every solution is eλtx0e^{\lambda t}\mathbf{x}_0. Otherwise AA is defective, with only one independent eigenvector v\mathbf{v}. In that case (A−λI)2=0(A - \lambda I)^2 = 0, so the column space of A−λIA - \lambda I is the line spanned by v\mathbf{v}, and there is always a vector w\mathbf{w} with (A−λI)w=v(A - \lambda I)\mathbf{w} = \mathbf{v}, called a generalized eigenvector. Then

x1=eλtv,x2=eλt(tv+w)\mathbf{x}_1 = e^{\lambda t}\mathbf{v}, \qquad \mathbf{x}_2 = e^{\lambda t}(t\mathbf{v} + \mathbf{w})

form a fundamental set of solutions. For an n×nn \times n matrix, an eigenvalue of multiplicity mm with fewer than mm independent eigenvectors produces solutions of the form eλte^{\lambda t} times a vector polynomial in tt of degree less than mm.

In plain words With only one eigen direction, the second solution needs a factor of tt, much like the repeated root case of a second-order equation.

Example For C=[11−13]C = \begin{bmatrix} 1 & 1 \\ -1 & 3 \end{bmatrix} the trace is 44 and the determinant is 44, so λ=2\lambda = 2 twice. Here C−2I=[−11−11]C - 2I = \begin{bmatrix} -1 & 1 \\ -1 & 1 \end{bmatrix}, so v=⟨1,1⟩\mathbf{v} = \langle 1, 1 \rangle, and w=⟨0,1⟩\mathbf{w} = \langle 0, 1 \rangle works. The second solution is x2=e2t⟨t,t+1⟩\mathbf{x}_2 = e^{2t}\langle t, t + 1 \rangle. Checking the first row, x1′=(2t+1)e2tx_1' = (2t + 1)e^{2t} and x1+x2=(t+t+1)e2tx_1 + x_2 = (t + t + 1)e^{2t}, which match.

Matrix exponential

Definition For any n×nn \times n matrix AA and real tt,

eAt=∑k=0∞(At)kk!=I+At+(At)22!+⋯ ,\begin{aligned} e^{At} &= \sum_{k=0}^{\infty} \frac{(At)^k}{k!} \\[0.4em] &= I + At + \frac{(At)^2}{2!} + \cdots, \end{aligned}

a series that converges for every AA and tt. It satisfies ddteAt=AeAt\frac{d}{dt}e^{At} = Ae^{At} and eA⋅0=Ie^{A \cdot 0} = I, so the unique solution of x′=Ax\mathbf{x}' = A\mathbf{x} with x(0)=x0\mathbf{x}(0) = \mathbf{x}_0 is x(t)=eAtx0\mathbf{x}(t) = e^{At}\mathbf{x}_0. If A=PΛP−1A = P\Lambda P^{-1} with Λ\Lambda diagonal, then eAt=PeΛtP−1e^{At} = Pe^{\Lambda t}P^{-1}. The rule eA+B=eAeBe^{A + B} = e^A e^B holds when AB=BAAB = BA and can fail otherwise.

In plain words The matrix exponential packs all solutions into one matrix. Multiply it by the starting state and you get the state at time tt.

Example For the defective matrix CC above, write C=2I+NC = 2I + N with N=C−2IN = C - 2I. The matrices 2I2I and NN commute and N2=0N^2 = 0, so the series for eNte^{Nt} stops after two terms and

eCt=e2t(I+tN)=e2t[1−tt−t1+t].\begin{aligned} e^{Ct} &= e^{2t}(I + tN) \\[0.4em] &= e^{2t}\begin{bmatrix} 1 - t & t \\ -t & 1 + t \end{bmatrix}. \end{aligned}

Phase portraits

Definition For a 2×22 \times 2 real system x′=Ax\mathbf{x}' = A\mathbf{x}, the phase plane is the x1x2x_1x_2 plane, and the phase portrait is the family of solution curves drawn there with arrows for increasing tt. If det⁡A≠0\det A \ne 0, the origin is the only equilibrium, and the eigenvalues classify it. A node, star node or spiral is stable when the real parts are negative and unstable when they are positive.

EigenvaluesOrigin
real, opposite signssaddle
real, distinct, same signnode
real, equal, A=λIA = \lambda Iproper (star) node
real, equal, A≠λIA \ne \lambda Idegenerate node
α±iβ\alpha \pm i\beta, β≠0\beta \ne 0, α≠0\alpha \ne 0spiral
±iβ\pm i\beta, β≠0\beta \ne 0center

In plain words Each solution is a path in the plane traced over time. Real eigenvalues give straight-line solutions along the eigenvectors, and complex ones make paths turn around the origin.

Example For B=[−12−2−1]B = \begin{bmatrix} -1 & 2 \\ -2 & -1 \end{bmatrix} the eigenvalues −1±2i-1 \pm 2i give a spiral. To find the turning direction, evaluate the velocity at ⟨1,0⟩\langle 1, 0 \rangle. It is B⟨1,0⟩=⟨−1,−2⟩B\langle 1, 0 \rangle = \langle -1, -2 \rangle, which points downward, so paths turn clockwise while the factor e−te^{-t} pulls them in.

Stability

Definition The equilibrium 0\mathbf{0} of x′=Ax\mathbf{x}' = A\mathbf{x} is stable if for every ε>0\varepsilon > 0 there is δ>0\delta > 0 such that ∥x(0)∥<δ\lVert \mathbf{x}(0) \rVert < \delta implies ∥x(t)∥<ε\lVert \mathbf{x}(t) \rVert < \varepsilon for all t≥0t \ge 0. It is asymptotically stable if it is stable and every solution tends to 0\mathbf{0} as t→∞t \to \infty, and unstable if it is not stable. For constant real AA,

  • if every eigenvalue has negative real part, 0\mathbf{0} is asymptotically stable,
  • if some eigenvalue has positive real part, 0\mathbf{0} is unstable,
  • if every real part is at most 00 and some is 00, then 0\mathbf{0} is stable but not asymptotically stable exactly when each eigenvalue with zero real part has as many independent eigenvectors as its multiplicity, and unstable otherwise.

In plain words Negative real parts mean every mode decays. A neutral mode with zero real part neither grows nor decays by itself, but if its eigenvalue is defective the extra factor tt makes it grow slowly.

Example The matrix N=[0100]N = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} has the double eigenvalue 00, so no mode grows exponentially. Yet eNt=I+tNe^{Nt} = I + tN and the solution from ⟨a,b⟩\langle a, b \rangle is ⟨a+bt,b⟩\langle a + bt, b \rangle, which is unbounded when b≠0b \ne 0. The origin is unstable. By contrast, [01−10]\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix} has eigenvalues ±i\pm i, its solutions are circles, and the origin is a stable center that is not asymptotically stable.

Check yourself

  1. Solve x′=[1221]x\mathbf{x}' = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix}\mathbf{x} with x(0)=⟨2,0⟩\mathbf{x}(0) = \langle 2, 0 \rangle and classify the origin.
Show the answer

The eigenvalues are 33 with ⟨1,1⟩\langle 1, 1 \rangle and −1-1 with ⟨1,−1⟩\langle 1, -1 \rangle. Matching x(0)\mathbf{x}(0) gives c1=c2=1c_1 = c_2 = 1, so

x(t)=e3t[11]+e−t[1−1].\mathbf{x}(t) = e^{3t}\begin{bmatrix} 1 \\ 1 \end{bmatrix} + e^{-t}\begin{bmatrix} 1 \\ -1 \end{bmatrix}.

The eigenvalues have opposite signs, so the origin is a saddle and it is unstable.

  1. Write y′′+2y′+5y=0y'' + 2y' + 5y = 0 as a system with x1=yx_1 = y and x2=y′x_2 = y', then classify the origin and its stability.
Show the answer

The system is x1′=x2x_1' = x_2 and x2′=−5x1−2x2x_2' = -5x_1 - 2x_2. The matrix has trace −2-2 and determinant 55, so λ2+2λ+5=0\lambda^2 + 2\lambda + 5 = 0 and λ=−1±2i\lambda = -1 \pm 2i. The origin is a spiral sink, asymptotically stable. At ⟨1,0⟩\langle 1, 0 \rangle the velocity is ⟨0,−5⟩\langle 0, -5 \rangle, so paths turn clockwise.

  1. Compute eAte^{At} for A=[3103]A = \begin{bmatrix} 3 & 1 \\ 0 & 3 \end{bmatrix} and solve x′=Ax\mathbf{x}' = A\mathbf{x} with x(0)=⟨1,2⟩\mathbf{x}(0) = \langle 1, 2 \rangle.
Show the answer

Write A=3I+NA = 3I + N with N=[0100]N = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} and N2=0N^2 = 0. Then

eAt=e3t[1t01],e^{At} = e^{3t}\begin{bmatrix} 1 & t \\ 0 & 1 \end{bmatrix},

and the solution is x(t)=e3t⟨1+2t,2⟩\mathbf{x}(t) = e^{3t}\langle 1 + 2t, 2 \rangle.

How this lesson connects

It also uses Matrices and linear maps, Scalar linear differential equations and Complex numbers and exponentials.