Direct Sums and Projections

Definition Let $U$ and $W$ be subspaces of a vector space $V$ over $\mathbb{F}$, where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. A decomposition of $v \in V$ along $U$ and $W$ is a way of writing $v = u + w$ with $u \in U$ and $w \in W$. Assume $U + W = V$, so that every vector of $V$ has at least one decomposition. Then every vector of $V$ has exactly one decomposition if and only if

The space $V$ need not be finite-dimensional. For the proof, suppose first that $U \cap W = \{0\}$ and $v = u_1 + w_1 = u_2 + w_2$. Then $u_1 - u_2 = w_2 - w_1$ lies in $U$ and in $W$, so it is $0$, which gives $u_1 = u_2$ and $w_1 = w_2$. Conversely, suppose decompositions are unique and let $c \in U \cap W$. Then $0 = 0 + 0$ and $0 = c + (-c)$ are both decompositions of $0$, with $c \in U$ and $-c \in W$. By uniqueness they agree, so $c = 0$. The hypothesis $U + W = V$ supplies existence and cannot be dropped. Zero intersection by itself only says that a vector has at most one decomposition.

Definition Let $U$ and $W$ be subspaces of a vector space $V$ over $\mathbb{F}$, where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. The space $V$ is the direct sum of $U$ and $W$, written

when $U + W = V$ and $U \cap W = \{0\}$. By the theorem on uniqueness of decomposition this means that every $v \in V$ can be written in exactly one way as $v = u + w$ with $u \in U$ and $w \in W$. Equivalently, the addition map from the product space $U \times W$ to $V$, sending $(u, w)$ to $u + w$, is a linear bijection. It is surjective exactly when $U + W = V$, and its kernel is the set of pairs $(c, -c)$ with $c \in U \cap W$.

If $V$ is finite-dimensional, the dimension formula

gives $\dim V = \dim U + \dim W$ for a direct sum, and a basis of $U$ followed by a basis of $W$ is a basis of $V$. Conversely, still with $V$ finite-dimensional, if $U \cap W = \{0\}$ and $\dim U + \dim W = \dim V$, then $U + W$ is a subspace of $V$ with the full dimension, so $U + W = V$ and the sum is direct.

Definition Let $U$ be a subspace of a vector space $V$ over $\mathbb{F}$, where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. A subspace $W$ of $V$ is a complement of $U$ in $V$, or complementary to $U$, if $V = U \oplus W$. The relation is symmetric, so $U$ is then also a complement of $W$. If $V$ is finite-dimensional, every subspace $U$ has a complement. One comes from extending a basis $u_1, \dots, u_k$ of $U$ to a basis $u_1, \dots, u_k, w_1, \dots, w_m$ of $V$ and taking

The full list spans $V$, so $U + W = V$, and a nonzero vector in both subspaces would give a nontrivial linear relation among the basis vectors, so $U \cap W = \{0\}$. In a finite-dimensional $V$ every complement of $U$ has dimension $\dim V - \dim U$. If $U = \{0\}$, its only complement is $V$, and if $U = V$, its only complement is $\{0\}$. Otherwise there are infinitely many. If $u \ne 0$ lies in $U$ and $w_1, \dots, w_m$ is a basis of one complement, then $\operatorname{span}\{w_1 + tu, w_2, \dots, w_m\}$ is a complement for every scalar $t$, and different values of $t$ give different complements.

Definition Let $V = U \oplus W$, where $V$ is a vector space over $\mathbb{F}$ and $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. For $v \in V$ let $v = u + w$ be its unique decomposition with $u \in U$ and $w \in W$. The projection onto $U$ along $W$ is the map $P \colon V \to V$ given by $P(v) = u$. It has the following properties.

1. The map $P$ is linear. If $v = u + w$ and $v' = u' + w'$ are decompositions and $a, b \in \mathbb{F}$, then

   $$\begin{aligned} av + bv' &= (au + bu') \\[0.4em] &\quad + (aw + bw') \end{aligned}$$

   with $au + bu' \in U$ and $aw + bw' \in W$. By uniqueness this is the decomposition of $av + bv'$, so $P(av + bv') = aP(v) + bP(v')$.

2. Every $u \in U$ satisfies $P(u) = u$ and every $w \in W$ satisfies $P(w) = 0$. Hence $\operatorname{im} P = U$ and $W \subseteq \ker P$. Conversely, if $P(v) = 0$, then the $U$ part of $v$ is $0$ and $v$ lies in $W$, so $\ker P = W$.

3. The square $P^2 = P \circ P$ equals $P$, because $P(v)$ already lies in $U$, where $P$ acts as the identity.

The map $I - P$, where $I$ is the identity map of $V$, sends $v$ to $w$ and is the projection onto $W$ along $U$. The projection depends on $W$ as well as on $U$. The same $U$ with a different complement gives a different map.

Definition Let $V$ be a vector space over $\mathbb{F}$, where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. A linear map $P \in \operatorname{Hom}(V, V)$ is idempotent if $P^2 = P$, that is, $P(P(v)) = P(v)$ for every $v \in V$. The theorem on idempotent maps says that if $P$ is idempotent, then

and $P(x) = x$ for every $x \in \operatorname{im} P$. So $P$ is the projection onto $\operatorname{im} P$ along $\ker P$. Since every projection along a complement is idempotent, the idempotent linear maps from $V$ to itself are exactly the projections along complements. No finite-dimensional hypothesis is needed.

For the proof, write every $v \in V$ as

The first term lies in $\operatorname{im} P$, and $P\big(v - P(v)\big) = P(v) - P^2(v) = 0$, so the second lies in $\ker P$. Hence $\operatorname{im} P + \ker P = V$. If $z \in \operatorname{im} P \cap \ker P$, write $z = P(x)$. Then $z = P(x) = P^2(x) = P(z) = 0$. Finally, for $x = P(y)$ in the image, $P(x) = P^2(y) = P(y) = x$.

An image and a kernel that form a direct sum do not by themselves make a map a projection. On any nonzero $V$ the map $T = 2I$ has image $V$ and kernel $\{0\}$, which form a direct sum, yet $T^2 = 4I \ne T$. A linear map $T$ from $V$ to itself is idempotent exactly when it fixes every vector of its image, since $T^2(v) = T\big(T(v)\big)$ and $T(v)$ lies in the image.

Definition Let $P$ be an idempotent real $n \times n$ matrix, so that $\mathbb{R}^n = \operatorname{im} P \oplus \ker P$, and use the dot product on $\mathbb{R}^n$. The matrix $P$ is the orthogonal projection onto $\operatorname{im} P$ when every vector of $\ker P$ is perpendicular to every vector of $\operatorname{im} P$, that is, when $\ker P = (\operatorname{im} P)^{\perp}$. The first condition says $\ker P \subseteq (\operatorname{im} P)^{\perp}$, and this inclusion is an equality because both subspaces have dimension $n - \dim \operatorname{im} P$. Otherwise $P$ is an oblique projection onto $\operatorname{im} P$ along $\ker P$. For an orthogonal projection the residual $v - Pv$ is perpendicular to $\operatorname{im} P$, and $Pv$ is the point of $\operatorname{im} P$ closest to $v$. Indeed, for every $u \in \operatorname{im} P$ the vector $Pv - u$ also lies in $\operatorname{im} P$, so the Pythagorean theorem gives

which is smallest exactly when $u = Pv$. An oblique projection loses that property for some vectors, because some of its residuals are not perpendicular to the image.

Being a projection uses no inner product at all, while being orthogonal depends on the dot product. An idempotent real matrix is an orthogonal projection exactly when it is symmetric. If $P^{\mathsf{T}} = P$, then for $x \in \operatorname{im} P$ and $y \in \ker P$,

Conversely, if $\ker P$ is perpendicular to $\operatorname{im} P$, splitting $v'$ as $Pv' + (v' - Pv')$ gives $(Pv) \cdot v' = (Pv) \cdot (Pv')$, and in the same way $v \cdot (Pv') = (Pv) \cdot (Pv')$. So $(Pv) \cdot v' = v \cdot (Pv')$ for all $v$ and $v'$, which means $P^{\mathsf{T}} = P$. In particular an idempotent real matrix that is not symmetric is always oblique. Over $\mathbb{C}$, with $\langle x, y \rangle = x^{*}y$ in place of the dot product, the same argument shows that an idempotent complex matrix is an orthogonal projection exactly when $P^{*} = P$. Later lessons extend this to general inner products.