The Moore-Penrose Conditions

Definition Let $A \in \mathbb{F}^{m \times n}$ and $X \in \mathbb{F}^{n \times m}$. The Moore-Penrose conditions on $X$ are

The first two are reconstruction identities. The last two say that the $m \times m$ matrix $AX$ and the $n \times n$ matrix $XA$ are Hermitian, which over $\mathbb{R}$ means symmetric. The pseudoinverse $A^{+} = V\Sigma^{+}U^{*}$ satisfies all four, for every shape and every rank, including $A = 0$. In singular coordinates $\Sigma\Sigma^{+}\Sigma = \Sigma$ and $\Sigma^{+}\Sigma\Sigma^{+} = \Sigma^{+}$ hold entry by entry, since $\sigma \cdot \sigma^{-1} \cdot \sigma = \sigma$ for each positive $\sigma$ and every other entry is zero. The products $\Sigma\Sigma^{+}$ and $\Sigma^{+}\Sigma$ are real diagonal, with $r$ ones followed by zeros, so they are Hermitian. Multiplying by the unitary factors keeps each identity, for instance

and $(AA^{+})^{*} = (U\Sigma\Sigma^{+}U^{*})^{*} = U\Sigma\Sigma^{+}U^{*}$. If $A$ is square and invertible, $X = A^{-1}$ satisfies all four.

Definition A square matrix $P \in \mathbb{F}^{k \times k}$ is idempotent if $P^2 = P$ and Hermitian if $P^{*} = P$. A Hermitian idempotent matrix is an orthogonal projector. Every $x$ splits as $x = Px + (x - Px)$ with $Px \in \operatorname{im} P$, and $x - Px \in \ker P$ by idempotence. Hermitian symmetry makes the two pieces orthogonal, since for all $x, y$

Hence $\ker P = (\operatorname{im} P)^{\perp}$, and $Px$ is the point of $\operatorname{im} P$ closest to $x$. Conversely, the orthogonal projector onto any subspace is Hermitian and idempotent. Such a projector is determined by its image alone, because it fixes the image and sends the orthogonal complement to zero. For the Moore-Penrose conditions this gives two facts. If $X$ satisfies (1) and (3), then $P = AX$ is Hermitian by (3) and idempotent since $P^2 = (AXA)X = P$. Every $Pb$ lies in $\operatorname{im} A$, and every $Av$ equals $AXAv = P(Av)$, so $P$ is the orthogonal projector onto $\operatorname{im} A$. If $X$ satisfies (1) and (4), then $Q = XA$ is Hermitian by (4) and idempotent since $Q^2 = X(AXA) = Q$. Its kernel is $\ker A$, so it is the orthogonal projector onto $(\ker A)^{\perp}$. To see the kernels agree, note that $Qv = 0$ exactly when $Av = 0$. Indeed $Av = 0$ gives $Qv = XAv = 0$, and $Qv = 0$ gives $Av = AXAv = AQv = 0$.

Definition The theorem says that for every $A \in \mathbb{F}^{m \times n}$ exactly one $X \in \mathbb{F}^{n \times m}$ satisfies all four Moore-Penrose conditions, namely $X = A^{+}$. In particular $A^{+}$ does not depend on the singular value decomposition used to build it. Existence was checked above. For uniqueness, let $X$ satisfy all four. By the facts about Hermitian idempotents, $AX = P$ is the orthogonal projector onto $\operatorname{im} A$ and $XA = Q$ is the orthogonal projector onto $(\ker A)^{\perp}$, and neither depends on $X$. Condition (2) gives $X = XAX = QX$, so every $Xb$ lies in $(\ker A)^{\perp}$. Also $A(Xb) = Pb$. The restriction of $A$ to $(\ker A)^{\perp}$ is injective, because a vector in both $\ker A$ and $(\ker A)^{\perp}$ is orthogonal to itself and so is zero. It maps onto $\operatorname{im} A$, because $Av = A(Qv)$ for every $v$, as $v - Qv \in \ker A$. So $Pb$ has exactly one preimage in $(\ker A)^{\perp}$, and $Xb$ has to be that preimage. Any two matrices satisfying the four conditions therefore agree on every $b$ and are equal, and $A^{+}$ is one of them.

Definition Let $A \in \mathbb{F}^{n \times n}$ be singular, of rank $r < n$. It has no inverse, but its pseudoinverse $A^{+} \in \mathbb{F}^{n \times n}$ exists and is the unique matrix satisfying the four conditions. The products $AA^{+}$ and $A^{+}A$ are orthogonal projectors of rank $r$, onto $\operatorname{im} A$ and onto $\operatorname{im} A^{*}$, so neither is $I_n$. They are equal exactly when $\operatorname{im} A = \operatorname{im} A^{*}$, which holds for normal matrices, since then $\ker A = \ker A^{*}$. For a normal matrix the spectral theorem over $\mathbb{C}$ gives $A = Q\Lambda Q^{*}$ with $Q \in \mathbb{C}^{n \times n}$ unitary and $\Lambda$ diagonal, even when $A$ is real, and then

where $\Lambda^{+}$ reciprocates the nonzero diagonal entries and keeps the zeros, because the four conditions hold for $\Lambda$ and $\Lambda^{+}$ entry by entry, conjugation by the unitary $Q$ keeps them, and only $A^{+}$ satisfies all four. In particular, the pseudoinverse of a diagonal matrix reciprocates its nonzero entries, signs included.

Definition Let $A \in \mathbb{F}^{m \times n}$. A matrix $X \in \mathbb{F}^{n \times m}$ with $AXA = A$, condition (1) alone, is a generalized inverse of $A$, and it is called reflexive if it also satisfies (2). Generalized inverses always exist, since $A^{+}$ is one, but they are unique only when $A$ is square and invertible, and then $X = A^{-1}$. Each one solves every consistent system. If $Ax_0 = b$, then

Adding (3) makes $AX$ the orthogonal projector onto $\operatorname{im} A$, so $Xb$ is a least-squares solution for every $b$. Adding (4) makes $XA$ the orthogonal projector onto $(\ker A)^{\perp}$, so for consistent $b$ the vector $Xb = XAx_0$ is the solution of minimum norm. Only all four conditions together give the minimum-norm least-squares solution $A^{+}b$ for every $b$.