The Matrix Exponential

Definition For a complex number $z$ the exponential series is

It converges absolutely for every $z \in \mathbb{C}$, and its sum is written $e^z$. The term with index $k + 1$ has size $\lvert z \rvert / (k + 1)$ times the size of the term with index $k$. Choose an integer $K \ge 0$ with $K + 1 \ge 2\lvert z \rvert$. Then for every $k \ge K$ the term with index $k + 1$ is at most half the size of the term with index $k$, so the tail is bounded by a geometric series,

So the increasing partial sums of $\sum \lvert z \rvert^k / k!$ are bounded and converge, the tails tend to $0$, and the partial sums of the original series form a Cauchy sequence. By the completeness of $\mathbb{C}$ they converge. Three facts are used later. The sum of the absolute values of the terms is $e^{\lvert z \rvert}$. For real $x \ge 0$ every term is nonnegative, so keeping a single term gives $e^x \ge x^m / m!$ for each $m \ge 0$. The product law $e^{z + w} = e^z e^w$ is the $1 \times 1$ case of the commuting rule below.

Definition For $A \in \mathbb{F}^{n \times n}$ and a scalar $t \in \mathbb{F}$, the matrix exponential is

the limit of the partial sums $S_K = \sum_{k=0}^{K} t^k A^k / k!$, which exists by the convergence theorem below. The case $t = 1$ is written $e^A$. Each partial sum is a polynomial in $A$, so $e^{tA}$ commutes with $A$, and it is real when $A$ and $t$ are real. For the zero matrix every term after the first vanishes, so $e^{0} = I$. If $A = \operatorname{diag}(d_1, \dots, d_n)$, every power is diagonal and $e^{tA} = \operatorname{diag}(e^{td_1}, \dots, e^{td_n})$. If $N^s = 0$, the series for $e^{tN}$ stops after the term with $N^{s-1}$. The definition needs no diagonalization, so defective matrices are included. Watch for this. The exponential of a matrix is not the matrix of exponentials of its entries.

For readers who know derivatives, $e^{tA}$ is differentiable in real $t$ with derivative $Ae^{tA} = e^{tA}A$, and $x(t) = e^{tA}x_0$ is the only differentiable solution of $x'(t) = Ax(t)$ with $x(0) = x_0$. Nothing else in this lesson depends on that fact.

Definition Let $\lVert \cdot \rVert$ be an induced matrix norm on $\mathbb{F}^{n \times n}$, the operator norm of a vector norm on $\mathbb{F}^n$, such as the spectral norm. Then $\lVert I \rVert = 1$ and $\lVert XY \rVert \le \lVert X \rVert \, \lVert Y \rVert$. For every $A \in \mathbb{F}^{n \times n}$ and every scalar $t$, the partial sums $S_K$ of the exponential series converge, entry by entry and in this norm, and

For the proof, induction gives $\lVert A^k \rVert \le \lVert A \rVert^k$, where the case $k = 0$ uses $\lVert I \rVert = 1$. For $L > K$,

which is part of the tail of the convergent scalar series for $e^{\lvert t \rvert \lVert A \rVert}$, so it tends to $0$ as $K \to \infty$. In finite dimension each entry of a matrix $X$ satisfies $\lvert x_{ij} \rvert \le c \lVert X \rVert$ for a constant $c$ that depends only on the norm, and $c = 1$ for the spectral norm. So every entry of $S_K$ is a Cauchy sequence of scalars and converges. Letting $L \to \infty$ shows that $\lVert e^{tA} - S_K \rVert$ is at most the scalar tail after $K$, and the bound $\lVert S_K \rVert \le e^{\lvert t \rvert \lVert A \rVert}$ passes to the limit. The argument uses no eigenvalues and no diagonalization.

Definition Let $A \in \mathbb{F}^{n \times n}$, let $P \in \mathbb{F}^{n \times n}$ be invertible and let $C = P^{-1}AP$. Then for every scalar $t$

Each partial sum satisfies $S_K(C) = P^{-1}S_K(A)P$, because polynomials respect similarity. Multiplication by fixed matrices is continuous, since

and the right side tends to $0$. So the exponential of an operator $T \colon V \to V$ on a finite-dimensional space is well defined, and it does not depend on the basis used to compute it. If $A = PDP^{-1}$ with $D = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$, then $e^{tA} = P\operatorname{diag}(e^{t\lambda_1}, \dots, e^{t\lambda_n})P^{-1}$. If $A = PJP^{-1}$ with $J$ a Jordan form over $\mathbb{C}$, then $e^{tA} = Pe^{tJ}P^{-1}$, and $e^{tJ}$ is block diagonal with the exponentials of the Jordan blocks, because every power of a block diagonal matrix is block diagonal.

Definition Let $A, C \in \mathbb{F}^{n \times n}$ commute, $AC = CA$. Then

For the proof, the double series $\sum_{j, k \ge 0} A^j C^k / (j!\,k!)$ converges absolutely, because the sum of the norms of its terms is at most $e^{\lVert A \rVert} e^{\lVert C \rVert}$. Its sums over the squares $j, k \le K$ are the products of the partial sums, which converge to $e^A e^C$. Absolute convergence allows the terms to be grouped by total degree $m = j + k$ without changing the sum, and because $A$ and $C$ commute the binomial theorem gives

Summing over $m$ gives $e^{A + C}$. Several consequences follow. For scalars $r$ and $t$, $e^{(r + t)A} = e^{rA}e^{tA}$. Since $e^A e^{-A} = e^{0} = I$, every matrix exponential is invertible, with inverse $e^{-A}$. The $1 \times 1$ case is the scalar law $e^{z + w} = e^z e^w$. For a Jordan block $J = \lambda I + N$ with $N^s = 0$, the parts $\lambda I$ and $N$ commute, so

exactly. The finite sum is not a truncation, since every later term is $0$.

Definition Let $A \in \mathbb{F}^{n \times n}$ and let $t \ge 0$ be real. If every eigenvalue of $A$ in $\mathbb{C}$ has negative real part, then $e^{tA} \to 0$ as $t \to \infty$, so every trajectory $x(t) = e^{tA}x_0$ tends to $0$. Transient growth means that some trajectory grows for a while all the same, that is $\lVert e^{tA}x_0 \rVert > \lVert x_0 \rVert$ for some $x_0$ and some $t > 0$, or equivalently $\lVert e^{tA} \rVert > 1$ for some $t > 0$ in the induced norm.

The decay statement needs two scalar facts, and neither uses calculus. First, for real $t$ and complex $\lambda$, conjugating the absolutely convergent series term by term gives $\overline{e^{t\lambda}} = e^{t\overline{\lambda}}$, so

A real exponential is positive, because $e^x = (e^{x/2})^2$ and $e^x e^{-x} = 1$, so $\lvert e^{t\lambda} \rvert = e^{t\operatorname{Re}\lambda}$. Second, for $a > 0$ and an integer $m \ge 0$, keeping one term of the series gives $e^{at} \ge (at)^{m+1}/(m+1)!$, so for $t > 0$

which tends to $0$. In a Jordan form $A = PJP^{-1}$ over $\mathbb{C}$, every entry of $e^{tJ}$ is $e^{t\lambda}$ times a polynomial in $t$, so with $a = -\operatorname{Re}\lambda > 0$ every entry tends to $0$, and so does every entry of $e^{tA} = Pe^{tJ}P^{-1}$. Negative real parts do not force $\lVert e^{tA}x_0 \rVert$ to decrease at every moment. For a normal matrix $A = U\Lambda U^{*}$ with $U$ unitary and $\Lambda = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$, $e^{tA} = Ue^{t\Lambda}U^{*}$ has spectral norm $\max_i e^{t\operatorname{Re}\lambda_i}$, so transient growth in the Euclidean norm requires a matrix that is not normal.