Bases and Dimension, linear algebra I lesson

The vector $(3, 2)$ is three steps along $(1, 0)$ plus two steps along $(0, 1)$ . It is also one step along $(1, 0)$ plus two steps along $(1, 1)$ . Both descriptions are correct, but the numbers mean something only once you know which vectors they multiply and in which order. A basis is a list of reference vectors that describes every vector of a space in exactly one way. This lesson builds bases by removing redundant vectors or adding missing directions, and shows that every basis of a space has the same size, its dimension.

Key terms

Basis

Definition Tuples such as $(1, 1)$ stand for column vectors. Let $W$ be a subspace of $\mathbb{R}^n$ . A basis of $W$ is a list of vectors in $W$ that is linearly independent and spans $W$ . Spanning means every vector of $W$ is a linear combination of the list, and independence means that combination is unique. So a list of vectors in $W$ is a basis of $W$ exactly when every vector of $W$ is a combination of the list in exactly one way. Whether a list is a basis depends on the space it is meant to describe, so always name that space.

In plain words A basis has enough vectors to reach everything in the space and no spare ones.

Example The list $(1, 0)$ , $(1, 1)$ is a basis of $\mathbb{R}^2$ . Solving

\begin{aligned} a(1, 0) + b(1, 1) &= (a + b,\ b) \\[0.4em] &= (x, y) \end{aligned}

gives $b = y$ and $a = x - y$ , exactly one answer for every $(x, y)$ . Both conditions matter. The list $(1, 0)$ , $(2, 0)$ neither spans $\mathbb{R}^2$ nor is independent. Adding $(0, 1)$ makes it span, but it is still dependent. The single vector $(1, 0)$ is independent but spans only the $x$ -axis, so it is a basis of the $x$ -axis and not of $\mathbb{R}^2$ .

Standard basis

Definition For $j = 1, \dots, n$ let $\mathbf{e}_j$ be the vector in $\mathbb{R}^n$ with $1$ in entry $j$ and $0$ in every other entry. The list $(\mathbf{e}_1, \dots, \mathbf{e}_n)$ is the standard basis of $\mathbb{R}^n$ . It is a basis because every $\mathbf{x} = (x_1, \dots, x_n)$ satisfies

\mathbf{x} = x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \cdots + x_n\mathbf{e}_n,

and comparing entries shows that no other coefficients work. The matrix with these columns is the identity matrix $I_n$ , which has a pivot in every column.

In plain words The ordinary entries of a vector are its coordinates relative to the standard basis.

Example In $\mathbb{R}^3$ the vector $(3, -1, 2)$ equals $3\mathbf{e}_1 - \mathbf{e}_2 + 2\mathbf{e}_3$ . The standard basis of $\mathbb{R}^3$ is not a basis of the plane $W$ given by $z = x + y$ . A basis of $W$ must consist of vectors in $W$ , and none of $\mathbf{e}_1$ , $\mathbf{e}_2$ , $\mathbf{e}_3$ satisfies $z = x + y$ .

Coordinate vector

Definition An ordered basis of a subspace $W$ is a basis whose vectors are taken in a fixed order, written $\mathcal{B} = (\mathbf{b}_1, \dots, \mathbf{b}_k)$ . Each $\mathbf{v}$ in $W$ can be written in exactly one way as $\mathbf{v} = c_1\mathbf{b}_1 + \cdots + c_k\mathbf{b}_k$ , and the coordinate vector of $\mathbf{v}$ relative to $\mathcal{B}$ is the column of these coefficients,

[\mathbf{v}]_{\mathcal{B}} = \begin{bmatrix} c_1 \\ \vdots \\ c_k \end{bmatrix}.

It has $k$ entries, one for each basis vector, even when $\mathbf{v}$ itself has $n$ entries. For an ordered basis of all of $\mathbb{R}^n$ , put the basis vectors in the columns of the $n \times n$ matrix $B = [\mathbf{b}_1 \ \cdots \ \mathbf{b}_n]$ . Then $\mathbf{v} = B[\mathbf{v}]_{\mathcal{B}}$ . Reordering the basis reorders the coordinates in the same way, while the vector itself does not move.

In plain words Coordinates are the weights on the basis vectors, read in the order of the basis. Only the standard basis gives every vector coordinates equal to its ordinary entries.

Example Let $\mathcal{B} = (\mathbf{b}_1, \mathbf{b}_2)$ with $\mathbf{b}_1 = (1, 0)$ and $\mathbf{b}_2 = (1, 1)$ . By the basis example, $(x, y) = (x - y)\mathbf{b}_1 + y\mathbf{b}_2$ , so $\mathbf{v} = (3, 2)$ has $[\mathbf{v}]_{\mathcal{B}} = (1, 2)$ . Multiplying by $B$ rebuilds the vector,

B[\mathbf{v}]_{\mathcal{B}} = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}.

In the reversed order $(\mathbf{b}_2, \mathbf{b}_1)$ the same vector has coordinates $(2, 1)$ . A basis of a plane gives two coordinates even inside $\mathbb{R}^3$ . With the ordered basis $\mathcal{C} = ((1, 0, 1), (0, 1, 1))$ of the plane $z = x + y$ , found below, the vector $(2, 5, 7) = 2(1, 0, 1) + 5(0, 1, 1)$ has $[(2, 5, 7)]_{\mathcal{C}} = (2, 5)$ .

Pruning a spanning list

Definition Let $\mathbf{v}_1, \dots, \mathbf{v}_k$ span a subspace $W$ of $\mathbb{R}^n$ . While the list is dependent, choose a dependence relation and remove one vector whose coefficient in it is nonzero. That vector is a combination of the others, so the shorter list still spans $W$ . Each step removes a vector, so after at most $k$ steps the list is independent and still spans $W$ , which makes it a basis of $W$ . In particular every finite spanning list of $W$ contains a basis of $W$ . In $\mathbb{R}^n$ the relations come from elimination on the matrix whose columns are the listed vectors.

In plain words Throw out redundant vectors one at a time, choosing only vectors that a relation shows to be redundant, until no redundancy is left.

Example The vectors $\mathbf{c}_1 = (1, 0, 1)$ , $\mathbf{c}_2 = (0, 1, 1)$ and $\mathbf{c}_3 = (1, 1, 2)$ lie in the plane $W$ given by $z = x + y$ and span it, since every $(x, y, x + y)$ equals $x\mathbf{c}_1 + y\mathbf{c}_2$ . They satisfy $\mathbf{c}_1 + \mathbf{c}_2 - \mathbf{c}_3 = \mathbf{0}$ . Every coefficient is nonzero, so any one of the three may go. Removing $\mathbf{c}_3$ leaves $\mathbf{c}_1, \mathbf{c}_2$ , and $a\mathbf{c}_1 + b\mathbf{c}_2 = (a, b, a + b)$ is $\mathbf{0}$ only when $a = b = 0$ . So $(\mathbf{c}_1, \mathbf{c}_2)$ is a basis of $W$ . Removing $\mathbf{c}_1$ instead gives a different basis, $(\mathbf{c}_2, \mathbf{c}_3)$ .

Extending an independent list

Definition Let $\mathbf{v}_1, \dots, \mathbf{v}_k$ be linearly independent vectors in a subspace $W$ of $\mathbb{R}^n$ , and let $\mathbf{w}$ be a vector of $W$ outside $\operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\}$ . Then the list $\mathbf{v}_1, \dots, \mathbf{v}_k, \mathbf{w}$ is still linearly independent. In any relation among the longer list the coefficient of $\mathbf{w}$ must be zero, since otherwise $\mathbf{w}$ would lie in the span of the others, and then the independence of the original list makes every other coefficient zero as well. As long as the list does not span $W$ , such a $\mathbf{w}$ exists, so the step can be repeated. It stops after at most $n - k$ steps, because an independent list in $\mathbb{R}^n$ has at most $n$ vectors, and the result is a basis of $W$ that contains the original list.

In plain words Add a direction of the target space that the list cannot reach yet. That never creates a redundancy, and you keep going until the list reaches the whole target space.

Example Here the target space is all of $\mathbb{R}^3$ . The independent pair $\mathbf{c}_1, \mathbf{c}_2$ spans only the plane $W$ , so it is not yet a basis of $\mathbb{R}^3$ . The vector $\mathbf{e}_3 = (0, 0, 1)$ lies outside that span, because $1 \ne 0 + 0$ , so $\mathbf{c}_1, \mathbf{c}_2, \mathbf{e}_3$ is independent. It also spans $\mathbb{R}^3$ , because

\begin{aligned} (x, y, z) &= x\,\mathbf{c}_1 + y\,\mathbf{c}_2 \\[0.4em] &\quad + (z - x - y)\,\mathbf{e}_3. \end{aligned}

So $(\mathbf{c}_1, \mathbf{c}_2, \mathbf{e}_3)$ is a basis of $\mathbb{R}^3$ . In this ordered basis the vector $(2, -1, 0)$ has coordinates $(2, -1, -1)$ .

Dimension

Definition Let $W$ be a subspace of $\mathbb{R}^n$ . If $W$ is spanned by $m$ vectors, then every linearly independent list in $W$ has at most $m$ vectors. Consequently any two bases of $W$ have the same number of vectors, since each is independent and each spans. This common number is the dimension of $W$ , written $\dim W$ .

The bound comes from elimination. Write each vector of an independent list $\mathbf{u}_1, \dots, \mathbf{u}_k$ in $W$ as a combination of the $m$ spanning vectors and collect the coefficients as the columns of an $m \times k$ matrix $C$ . If $k > m$ , some column of $C$ has no pivot, since $C$ has only $m$ rows, so $C\mathbf{h} = \mathbf{0}$ for some nonzero $\mathbf{h}$ . The combination $h_1\mathbf{u}_1 + \cdots + h_k\mathbf{u}_k$ is the combination of the spanning vectors with coefficients $C\mathbf{h} = \mathbf{0}$ , so it equals $\mathbf{0}$ . That is a nontrivial relation among independent vectors, which is impossible.

The empty list is independent, since it allows no nontrivial relation, and its span is $\{\mathbf{0}\}$ , since its only combination, the empty sum, equals $\mathbf{0}$ . So the zero subspace has the empty list as its basis, and $\dim \{\mathbf{0}\} = 0$ . Every subspace of $\mathbb{R}^n$ has a basis, obtained by extending the empty list, and $\dim W \le n$ , with equality only for $W = \mathbb{R}^n$ . In $\mathbb{R}^n$ a list of exactly $n$ vectors is a basis as soon as it is independent, and also as soon as it spans.

In plain words Dimension counts independent directions. It belongs to the space, not to the basis you happen to pick, and it can be smaller than the number of entries the vectors have.

Example The plane $W$ given by $z = x + y$ has the basis $(\mathbf{c}_1, \mathbf{c}_2)$ , so $\dim W = 2$ , even though its vectors have three entries. The standard basis gives $\dim \mathbb{R}^3 = 3$ , and a line through the origin, spanned by one nonzero vector, has dimension $1$ . Any three vectors in $W$ , such as $\mathbf{c}_1, \mathbf{c}_2, \mathbf{c}_3$ , must be dependent, because three is more than $\dim W = 2$ .

Check yourself

Let $\mathcal{B} = ((2, 1), (1, 1))$ . Find $[\mathbf{v}]_{\mathcal{B}}$ for $\mathbf{v} = (3, 1)$ , check it by rebuilding $\mathbf{v}$ , and give the coordinates of $\mathbf{v}$ in the reversed order $((1, 1), (2, 1))$ .

Show the answer

Solve $2a + b = 3$ and $a + b = 1$ . Subtracting gives $a = 2$ , and then $b = -1$ . So $[\mathbf{v}]_{\mathcal{B}} = (2, -1)$ , and indeed

2\,(2, 1) - (1, 1) = (3, 1).

In the reversed order the entries swap, giving $(-1, 2)$ .

The vectors $(1, 1, 0)$ , $(0, 1, 1)$ and $(1, 0, 1)$ are linearly independent. Explain why they form a basis of $\mathbb{R}^3$ without checking that they span, and find the coordinates of $(2, 0, 0)$ in this ordered basis.

Show the answer

The space $\mathbb{R}^3$ has dimension $3$ , and in $\mathbb{R}^3$ a list of exactly three independent vectors is a basis. Two independent vectors would not be enough, and four vectors would be dependent. For the coordinates, solve

\begin{aligned} &c_1(1, 1, 0) + c_2(0, 1, 1) \\[0.4em] &\quad + c_3(1, 0, 1) = (2, 0, 0), \end{aligned}

that is $c_1 + c_3 = 2$ , $c_1 + c_2 = 0$ and $c_2 + c_3 = 0$ . The last two give $c_2 = -c_1$ and $c_3 = c_1$ , so $2c_1 = 2$ . The coordinates are $(1, -1, 1)$ , and $(1, 1, 0) - (0, 1, 1) + (1, 0, 1) = (2, 0, 0)$ confirms them.

Let $H$ be the plane $x - y + 2z = 0$ in $\mathbb{R}^3$ , with $\mathbf{p} = (1, 1, 0)$ , $\mathbf{q} = (-2, 0, 1)$ and $\mathbf{r} = (-1, 1, 1)$ . Is $\mathbf{p}, \mathbf{q}, \mathbf{r}$ a basis of $H$ ? If not, prune it to a basis, state $\dim H$ , and then extend your basis to a basis of $\mathbb{R}^3$ .

Show the answer

All three vectors satisfy the equation of $H$ . But $\mathbf{p} + \mathbf{q} = (-1, 1, 1) = \mathbf{r}$ , so the list is dependent and is not a basis. Remove $\mathbf{r}$ . The combination $a\mathbf{p} + b\mathbf{q} = (a - 2b,\ a,\ b)$ is zero only when $a = b = 0$ , so $\mathbf{p}, \mathbf{q}$ is independent. It spans $H$ , because a vector of $H$ has $x = y - 2z$ and therefore

(y - 2z,\ y,\ z) = y\,\mathbf{p} + z\,\mathbf{q}.

So $(\mathbf{p}, \mathbf{q})$ is a basis of $H$ and $\dim H = 2$ . To extend, pick a vector outside $H$ , such as $\mathbf{e}_1 = (1, 0, 0)$ , since $1 - 0 + 2 \cdot 0 = 1 \ne 0$ . The list $\mathbf{p}, \mathbf{q}, \mathbf{e}_1$ is then independent, and three independent vectors in $\mathbb{R}^3$ form a basis. Explicitly,

\begin{aligned} (x, y, z) &= y\,\mathbf{p} + z\,\mathbf{q} \\[0.4em] &\quad + (x - y + 2z)\,\mathbf{e}_1. \end{aligned}

Bases and Dimension

Key terms

Basis

Standard basis

Coordinate vector

Pruning a spanning list

Extending an independent list

Dimension

Check yourself

How this lesson connects

Builds on

Used later in

Where this leads in Linear Algebra II