Null Space and Left Null Space, linear algebra I lesson

Let $M$ be the 3 by 3 matrix with columns $(1, 2, 3)$ , $(2, 4, 6)$ and $(1, 0, 1)$ , where each tuple stands for a column vector. It sends both $(0, 0, 1)$ and $(2, -1, 1)$ to the same output $(1, 0, 1)$ , so the input change $(2, -1, 0)$ is invisible to it. Which input changes does a matrix send to zero, and which combinations of output entries vanish for every input? This lesson finds both sets, the null space and the left null space, counts their dimensions, and places all four fundamental subspaces of a matrix in their proper spaces.

Key terms

Null space

Definition The null space of a real $m \times n$ matrix $A$ is the solution set of the homogeneous system $A\mathbf{x} = \mathbf{0}$ ,

\operatorname{Null} A = \{\mathbf{x} \in \mathbb{R}^n \mid A\mathbf{x} = \mathbf{0}\}.

It is a subspace of $\mathbb{R}^n$ , the input space of $A$ . Row operations do not change it. If they take $A$ to $F$ , then $F = EA$ with $E$ invertible, so $A\mathbf{x} = \mathbf{0}$ exactly when $F\mathbf{x} = \mathbf{0}$ . A basis comes from the reduced row echelon form, by solving for the pivot variables in terms of the free variables. If $A\mathbf{p} = \mathbf{b}$ , then the solutions of $A\mathbf{x} = \mathbf{b}$ are exactly the vectors $\mathbf{p} + \mathbf{z}$ with $\mathbf{z}$ in $\operatorname{Null} A$ .

In plain words The null space holds the inputs the matrix sends to zero, which are exactly the input changes that leave every output as it was. The word null describes the output, not the inputs. The null space contains nonzero vectors exactly when the columns of $A$ are dependent.

Example The matrix $M$ and its reduced row echelon form are

\begin{aligned} M &= \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 0 \\ 3 & 6 & 1 \end{bmatrix}, \\[0.4em] F &= \begin{bmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix}. \end{aligned}

So $M\mathbf{x} = \mathbf{0}$ means $x_1 + 2x_2 = 0$ and $x_3 = 0$ . The variable $x_2 = t$ is free, and

\mathbf{x} = \begin{bmatrix} -2t \\ t \\ 0 \end{bmatrix} = t \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix}.

Hence $\operatorname{Null} M$ is the line through $\mathbf{0}$ spanned by $(-2, 1, 0)$ , and the rows of $M$ check it, since $-2 + 2 = 0$ , $-4 + 4 = 0$ and $-6 + 6 = 0$ . The inputs with output $(1, 0, 1)$ are the vectors $(0, 0, 1) + t(-2, 1, 0)$ , a parallel line that misses the origin.

Nullity

Definition The nullity of $A$ is $\dim \operatorname{Null} A$ . If $A$ is a real $m \times n$ matrix of rank $r$ , then

\dim \operatorname{Null} A = n - r.

Proof. An echelon form of $A$ has $r$ pivot columns, so $A\mathbf{x} = \mathbf{0}$ has $n - r$ free variables. For each free variable, set it to $1$ and the other free variables to $0$ , and solve for the pivot variables. This gives $n - r$ vectors in $\operatorname{Null} A$ . Each one has a $1$ in its own free position and $0$ in the other free positions, so a zero combination of them forces every coefficient to be $0$ , and they are independent. They also span. Subtracting from a solution the combination of them whose coefficients are its own free entries leaves a solution whose free entries are all $0$ , and the reduced equations then make its pivot entries $0$ as well. When $r = n$ there are no free variables, the null space is $\{\mathbf{0}\}$ , and its basis is the empty list.

In plain words Each free variable gives one independent direction of input changes that the matrix cannot detect. The nullity counts these directions, and the rank counts the pivot variables, the rest of the $n$ input coordinates.

Example Let

B = \begin{bmatrix} 1 & 3 & 1 & 1 \\ 2 & 6 & 1 & 3 \end{bmatrix}.

The operation $R_2 \to R_2 - 2R_1$ gives $(0, 0, -1, 1)$ , and then $R_2 \to -R_2$ and $R_1 \to R_1 - R_2$ give the reduced rows $(1, 3, 0, 2)$ and $(0, 0, 1, -1)$ . The pivots are in columns 1 and 3, so $\operatorname{rank} B = 2$ and the variables $x_2$ , $x_4$ are free. Setting $x_2 = 1$ , $x_4 = 0$ gives $(-3, 1, 0, 0)$ , and setting $x_2 = 0$ , $x_4 = 1$ gives $(-2, 0, 1, 1)$ . These two vectors form a basis of $\operatorname{Null} B$ , and the nullity is $4 - 2 = 2$ .

Left null space

Definition The left null space of a real $m \times n$ matrix $A$ is the null space of its transpose,

\begin{aligned} \operatorname{Null} A^{\top} = \{ & \mathbf{y} \in \mathbb{R}^m \\[0.4em] & \mid A^{\top}\mathbf{y} = \mathbf{0}\}, \end{aligned}

a subspace of $\mathbb{R}^m$ , the output space of $A$ . Transposing $A^{\top}\mathbf{y} = \mathbf{0}$ gives $\mathbf{y}^{\top}A = \mathbf{0}^{\top}$ , with $\mathbf{y}$ on the left, which explains the name. So a vector $\mathbf{y}$ is in the left null space exactly when the combination $y_1$ times row 1 plus $\cdots$ plus $y_m$ times row $m$ of $A$ is the zero row. For every output $A\mathbf{x}$ ,

\mathbf{y}^{\top}(A\mathbf{x}) = (\mathbf{y}^{\top}A)\mathbf{x} = 0.

Hence if $\mathbf{y}^{\top}\mathbf{b} \ne 0$ for some $\mathbf{y}$ in $\operatorname{Null} A^{\top}$ , the system $A\mathbf{x} = \mathbf{b}$ has no solution.

In plain words A left null vector is a list of weights on the rows that cancels them completely. The same weights give a measurement of the output that is zero for every input, so a target with a nonzero measurement cannot be reached.

Example To solve $M^{\top}\mathbf{y} = \mathbf{0}$ , reduce

M^{\top} = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 0 & 1 \end{bmatrix}.

The operation $R_2 \to R_2 - 2R_1$ gives a zero row and $R_3 \to R_3 - R_1$ gives $(0, -2, -2)$ . Swapping these two rows, then $R_2 \to -\tfrac{1}{2}R_2$ and $R_1 \to R_1 - 2R_2$ , leaves the reduced rows $(1, 0, 1)$ and $(0, 1, 1)$ . So $y_1 = -y_3$ and $y_2 = -y_3$ , and with $y_3 = s$ every solution is $s(-1, -1, 1)$ . The left null space of $M$ has basis $(-1, -1, 1)$ , which says that minus row 1 minus row 2 plus row 3 of $M$ is the zero row. Every output $\mathbf{w} = M\mathbf{x}$ therefore satisfies $-w_1 - w_2 + w_3 = 0$ . The target $(1, 0, 2)$ gives $-1 - 0 + 2 = 1$ , so $M\mathbf{x} = (1, 0, 2)$ has no solution.

Left nullity

Definition The left nullity of $A$ is $\dim \operatorname{Null} A^{\top}$ . If $A$ is a real $m \times n$ matrix of rank $r$ , then

\dim \operatorname{Null} A^{\top} = m - r.

This is the nullity count applied to the $n \times m$ matrix $A^{\top}$ , which has $m$ columns and rank $r$ , because $\operatorname{rank} A^{\top} = \operatorname{rank} A$ . The left nullity differs from the nullity $n - r$ whenever $m \ne n$ . If $\mathbf{y}_1, \dots, \mathbf{y}_{m-r}$ is a basis of $\operatorname{Null} A^{\top}$ , then $\mathbf{b}$ lies in $\operatorname{Col} A$ exactly when $\mathbf{y}_i^{\top}\mathbf{b} = 0$ for every $i$ . When $r = m$ there are no tests, and indeed $\operatorname{Col} A$ has dimension $m$ , so it is all of $\mathbb{R}^m$ . When $r < m$ , the vectors $\mathbf{b}$ that pass all these tests form the null space of the $(m - r) \times m$ matrix with the independent rows $\mathbf{y}_i^{\top}$ . That null space has dimension $m - (m - r) = r$ and contains $\operatorname{Col} A$ , which also has dimension $r$ , so the two subspaces are equal.

In plain words The left nullity counts the independent conditions that every reachable output must satisfy. When it is zero there are no conditions, and every target in $\mathbb{R}^m$ can be reached.

Example The matrix

A = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \end{bmatrix}

has $m = 2$ , $n = 3$ and rank $2$ . Its nullity is $3 - 2 = 1$ , with basis $(-1, -1, 1)$ , and its left nullity is $2 - 2 = 0$ . Directly, the system $A^{\top}\mathbf{y} = \mathbf{0}$ reads $y_1 = 0$ , $y_2 = 0$ and $y_1 + y_2 = 0$ , so only $\mathbf{y} = \mathbf{0}$ remains. The left null space is the zero subspace of $\mathbb{R}^2$ , whose basis is the empty list, not the list containing $\mathbf{0}$ . Every $\mathbf{b}$ in $\mathbb{R}^2$ is an output, even though one input direction is lost. For $M$ the left nullity is $3 - 2 = 1$ , so the single test $-w_1 - w_2 + w_3 = 0$ decides exactly which targets are reachable.

Ambient space of a subspace

Definition A subspace $W$ of $\mathbb{R}^k$ has ambient space $\mathbb{R}^k$ , the space whose vectors it is made of. Its vectors, including the vectors of any basis of $W$ , have $k$ entries, and $\dim W \le k$ . For a real $m \times n$ matrix $A$ , the subspaces $\operatorname{Row} A$ and $\operatorname{Null} A$ have ambient space $\mathbb{R}^n$ , the input space, while $\operatorname{Col} A$ and $\operatorname{Null} A^{\top}$ have ambient space $\mathbb{R}^m$ , the output space. Subspaces with different ambient spaces cannot be equal, whatever their dimensions.

In plain words Before asking whether a vector lies in a subspace, check that it has the right number of entries. The number of entries says where a vector lives, and the dimension says how much room the subspace takes up there.

Example The matrix

A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix}

has rank $1$ . Its column space is the line in $\mathbb{R}^2$ spanned by $(1, 2)$ , and its row space is the line in $\mathbb{R}^3$ spanned by $(1, 2, 3)$ . Both have dimension $1$ , yet they are different spaces. The null space is the plane $x_1 + 2x_2 + 3x_3 = 0$ in $\mathbb{R}^3$ , with basis $(-2, 1, 0)$ , $(-3, 0, 1)$ , and the left null space is the line in $\mathbb{R}^2$ spanned by $(-2, 1)$ . That vector is a left null vector, since $-2$ times row 1 plus row 2 is the zero row. The product of $A$ with $(-2, 1)$ is not even defined, because $A$ takes inputs with three entries.

Four fundamental subspaces

Definition The four fundamental subspaces of a real $m \times n$ matrix $A$ of rank $r$ are its column space, left null space, row space and null space.

Subspace	Ambient space	Dimension
$\operatorname{Col} A$	$\mathbb{R}^m$	$r$
$\operatorname{Null} A^{\top}$	$\mathbb{R}^m$	$m - r$
$\operatorname{Row} A$	$\mathbb{R}^n$	$r$
$\operatorname{Null} A$	$\mathbb{R}^n$	$n - r$

The dimensions in the output space add up to $m$ , and those in the input space add up to $n$ . Elimination gives all four bases. Take the pivot columns of $A$ for $\operatorname{Col} A$ , the nonzero rows of an echelon form for $\operatorname{Row} A$ , and the free-variable solutions of $A\mathbf{x} = \mathbf{0}$ and of $A^{\top}\mathbf{y} = \mathbf{0}$ for the two null spaces.

In plain words Two of the spaces describe outputs, what the matrix can reach and which conditions every output obeys. The other two describe inputs, what the rows can detect and which changes the matrix cannot see. Once the size of the matrix is known, one number, the rank, fixes all four dimensions.

Example For $M$ we have $m = n = 3$ and $r = 2$ .

Subspace	Basis	Dimension
$\operatorname{Col} M$	$(1, 2, 3)$ , $(1, 0, 1)$	$2$
$\operatorname{Null} M^{\top}$	$(-1, -1, 1)$	$1$
$\operatorname{Row} M$	$(1, 2, 0)$ , $(0, 0, 1)$	$2$
$\operatorname{Null} M$	$(-2, 1, 0)$	$1$

All four lie in $\mathbb{R}^3$ only because $M$ is square. The first two are subspaces of the output space and the last two of the input space, and each basis is checked in its own role. The basis vector of the null space satisfies $M\mathbf{x} = \mathbf{0}$ , the basis vector of the left null space satisfies $\mathbf{y}^{\top}M = \mathbf{0}^{\top}$ , and the column basis consists of columns of $M$ itself.

Check yourself

Find a basis of $\operatorname{Null} A$ and a basis of $\operatorname{Null} A^{\top}$ for the matrix below.

A = \begin{bmatrix} 1 & -3 \\ -2 & 6 \end{bmatrix}

Show the answer

The second row is $-2$ times the first, so $A\mathbf{x} = \mathbf{0}$ reduces to $x_1 - 3x_2 = 0$ . With $x_2 = t$ free, the null space has basis $(3, 1)$ . The transpose has rows $(1, -2)$ and $(-3, 6)$ , so $A^{\top}\mathbf{y} = \mathbf{0}$ reduces to $y_1 - 2y_2 = 0$ , and the left null space has basis $(2, 1)$ . Check that $A$ times $(3, 1)$ is $(3 - 3, -6 + 6) = (0, 0)$ , and that $2$ times row 1 plus row 2 is $(2 - 2, -6 + 6)$ , the zero row. Even for a square matrix the two null spaces can differ.

A real 4 by 6 matrix $A$ has rank $4$ . Give the ambient space and the dimension of each of its four fundamental subspaces. Does $A\mathbf{x} = \mathbf{b}$ have a solution for every $\mathbf{b}$ , and can a solution be unique?

Show the answer

The column space lies in $\mathbb{R}^4$ with dimension $4$ , and the left null space lies in $\mathbb{R}^4$ with dimension $4 - 4 = 0$ , so it is $\{\mathbf{0}\}$ with the empty basis. The row space lies in $\mathbb{R}^6$ with dimension $4$ , and the null space lies in $\mathbb{R}^6$ with dimension $6 - 4 = 2$ . A subspace of $\mathbb{R}^4$ of dimension $4$ is all of $\mathbb{R}^4$ , so every $\mathbf{b}$ is reachable. No solution is unique, because each solution plus any nonzero vector of the two-dimensional null space is another solution.

For the matrix below, find a basis of the left null space and the nullity. Then decide, without solving for an input, whether $\mathbf{c} = (1, 0, 1)$ and $\mathbf{b} = (1, 1, 1)$ lie in $\operatorname{Col} A$ .

A = \begin{bmatrix} 1 & 2 \\ 2 & 1 \\ 0 & 3 \end{bmatrix}

Show the answer

The transpose has rows $(1, 2, 0)$ and $(2, 1, 3)$ . The operation $R_2 \to R_2 - 2R_1$ gives $(0, -3, 3)$ , then $R_2 \to -\tfrac{1}{3}R_2$ gives $(0, 1, -1)$ and $R_1 \to R_1 - 2R_2$ gives $(1, 0, 2)$ . So $y_1 = -2y_3$ and $y_2 = y_3$ , and the single vector $\mathbf{y} = (-2, 1, 1)$ forms a basis of the left null space. Indeed $-2$ times row 1 plus row 2 plus row 3 is $(-2 + 2 + 0, -4 + 1 + 3) = (0, 0)$ . The transpose has two pivots, so $\operatorname{rank} A = 2$ and the nullity of $A$ is $2 - 2 = 0$ . For $\mathbf{c}$ the test gives $\mathbf{y}^{\top}\mathbf{c} = -2 + 0 + 1 = -1 \ne 0$ , so $\mathbf{c}$ is not in $\operatorname{Col} A$ . For $\mathbf{b}$ it gives $-2 + 1 + 1 = 0$ . The left nullity is $3 - 2 = 1$ , so this single test is exact and $\mathbf{b}$ is in $\operatorname{Col} A$ . In fact the input $(\tfrac{1}{3}, \tfrac{1}{3})$ gives the output $(1, 1, 1)$ .

Null Space and Left Null Space

Key terms

Null space

Nullity

Left null space

Left nullity

Ambient space of a subspace

Four fundamental subspaces

Check yourself

How this lesson connects

Builds on

Used later in

Where this leads in Linear Algebra II