Linear Combinations and Span, linear algebra I lesson

Suppose you may move only along two fixed vectors, $\mathbf{p} = (1, 0)$ and $\mathbf{q} = (1, 1)$ , using any real amount of each. Here and below a tuple stands for a column vector. Can you reach $(3, 2)$ ? Yes, because $\mathbf{p} + 2\mathbf{q} = (3, 2)$ , but the more useful question is which targets can be reached at all, and how to prove an answer either way. This lesson names the combinations you can form, describes the set of everything they reach, and gives a reliable test for whether a given target belongs to it.

Key terms

Linear combination

Definition Let $\mathbf{v}_1, \dots, \mathbf{v}_k$ be vectors in $\mathbb{R}^n$ and let $a_1, \dots, a_k$ be real numbers. The vector

a_1\mathbf{v}_1 + a_2\mathbf{v}_2 + \cdots + a_k\mathbf{v}_k

is a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_k$ , and the numbers $a_1, \dots, a_k$ are its coefficients. Each coefficient $a_j$ multiplies the whole vector $\mathbf{v}_j$ . Coefficients may be any real numbers, including zero, negative numbers and fractions. All the vectors must lie in the same $\mathbb{R}^n$ , so that the sum is defined.

In plain words Take some amount of each given vector and add the results. The coefficients are the amounts.

Example Let $\mathbf{u} = (2, 1)$ and $\mathbf{v} = (1, 3)$ . For coefficients $a$ and $b$ ,

a\mathbf{u} + b\mathbf{v} = \begin{bmatrix} 2a + b \\ a + 3b \end{bmatrix}.

The coefficients $a = 1$ and $b = 2$ give $\mathbf{u} + 2\mathbf{v} = (4, 7)$ , and $a = 1$ and $b = -1$ give $\mathbf{u} - \mathbf{v} = (1, -2)$ . Keep the two descriptions of the first result apart. The coefficients $1$ and $2$ say how much of each vector was used, while $4$ and $7$ are the entries of the result.

Span

Definition The span of vectors $\mathbf{v}_1, \dots, \mathbf{v}_k$ in $\mathbb{R}^n$ is the set of all their linear combinations,

\begin{aligned} &\operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\} \\[0.4em] &\quad = \{\, a_1\mathbf{v}_1 + \cdots + a_k\mathbf{v}_k \mid \\[0.4em] &\qquad\quad a_1, \dots, a_k \in \mathbb{R} \,\}. \end{aligned}

It is a subset of $\mathbb{R}^n$ . It always contains $\mathbf{0}$ , by taking every coefficient equal to $0$ , and it contains each $\mathbf{v}_j$ , by taking $a_j = 1$ and the other coefficients $0$ . The span of one nonzero vector is the line through the origin in its direction. If every $\mathbf{v}_j$ is the zero vector, the span is $\{\mathbf{0}\}$ , a set with one element.

In plain words The span is everything you can reach with the given vectors using any real amounts. Because the amounts can be negative or very large, the span contains the whole line through the origin along each nonzero given vector, not only the arrow out to its tip.

Example The span of the single vector $(1, 2)$ is the set of all vectors $t(1, 2) = (t, 2t)$ with $t$ real, the line $y = 2x$ through the origin. It contains $-3(1, 2) = (-3, -6)$ and $5(1, 2) = (5, 10)$ , neither of which lies on the segment from the origin to $(1, 2)$ . It does not contain $(1, 1)$ . The first entry would force $t = 1$ , and then the second entry is $2$ , not $1$ .

Generating vectors

Definition If $S = \operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\}$ , we say that $\mathbf{v}_1, \dots, \mathbf{v}_k$ span $S$ , or generate $S$ , and we call them generating vectors, or generators, of $S$ . To prove that given vectors span a set $S$ , show two inclusions. First, every linear combination of the vectors lies in $S$ . Second, every vector of $S$ is a linear combination of them, which is shown by giving coefficients that work for an arbitrary vector of $S$ .

In plain words Generators are the building blocks, and the span is everything built from them. To show that some vectors reach exactly a given set, check that nothing outside the set is reached and that nothing inside it is missed.

Example Let $\mathbf{c}_1 = (1, 0, 1)$ and $\mathbf{c}_2 = (0, 1, 1)$ , and let $S$ be the plane $z = x + y$ in $\mathbb{R}^3$ . Every combination

a\mathbf{c}_1 + b\mathbf{c}_2 = \begin{bmatrix} a \\ b \\ a + b \end{bmatrix}

has third entry equal to the sum of the first two, so it lies in $S$ . Conversely, a vector in $S$ has the form $(x, y, x + y)$ , and it equals $x\mathbf{c}_1 + y\mathbf{c}_2$ . So $\mathbf{c}_1$ and $\mathbf{c}_2$ span the plane $S$ , which is only part of $\mathbb{R}^3$ . In the same way $\mathbf{p} = (1, 0)$ and $\mathbf{q} = (1, 1)$ span all of $\mathbb{R}^2$ , because every $(x, y)$ equals $(x - y)\mathbf{p} + y\mathbf{q}$ .

Membership in a span

Definition A vector $\mathbf{b}$ in $\mathbb{R}^n$ belongs to $\operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\}$ exactly when there are real numbers $x_1, \dots, x_k$ with

x_1\mathbf{v}_1 + \cdots + x_k\mathbf{v}_k = \mathbf{b}.

Comparing entries turns this vector equation into $n$ linear equations in the $k$ unknown coefficients, and $\mathbf{b}$ is in the span exactly when these equations have a solution. A yes answer is settled by giving coefficients and checking every entry. A no answer needs a reason why no coefficients at all can work, for example equations that contradict each other.

In plain words To show that a target is reachable, give the recipe and check it. To show that it is not, prove that every recipe fails, not only the ones you tried.

Example With $\mathbf{c}_1 = (1, 0, 1)$ and $\mathbf{c}_2 = (0, 1, 1)$ , the equation $a\mathbf{c}_1 + b\mathbf{c}_2 = (3, -2, 1)$ says $a = 3$ , $b = -2$ and $a + b = 1$ . The first two equations fix the coefficients and the third holds, so $(3, -2, 1) = 3\mathbf{c}_1 - 2\mathbf{c}_2$ is in the span. For the target $(3, -2, 4)$ the first two equations again force $a = 3$ and $b = -2$ , but then $a + b = 1 \ne 4$ . No coefficients work, so $(3, -2, 4)$ is not in the span. If the generator $\mathbf{e}_3 = (0, 0, 1)$ is added with a coefficient $c$ , the third equation becomes $a + b + c = 4$ , and $c = 3$ gives

3\mathbf{c}_1 - 2\mathbf{c}_2 + 3\mathbf{e}_3 = \begin{bmatrix} 3 \\ -2 \\ 4 \end{bmatrix}.

With all three generators every $(x, y, z)$ is reached, by $a = x$ , $b = y$ and $c = z - x - y$ , so they span $\mathbb{R}^3$ .

Redundant generator

Definition In a list $\mathbf{v}_1, \dots, \mathbf{v}_k$ of at least two vectors, a vector $\mathbf{v}_j$ is a redundant generator if it is a linear combination of the other vectors in the list. Removing a redundant generator does not change the span. In any combination of the full list, replace $\mathbf{v}_j$ by its expression in the others and collect terms. The result is a combination of the shorter list. Conversely, a combination of the shorter list is a combination of the full list with coefficient $0$ on $\mathbf{v}_j$ . The zero vector in such a list is always redundant, since it equals $0$ times any other vector of the list. When a list contains a redundant generator, every vector of its span can be written with more than one choice of coefficients.

In plain words A redundant generator adds a new arrow but no new reach. Everything it could contribute can already be built from the others.

Example Add $\mathbf{c}_3 = \mathbf{c}_1 + \mathbf{c}_2 = (1, 1, 2)$ to the generators of the plane $z = x + y$ . It points in a new direction, yet

\begin{aligned} &a\mathbf{c}_1 + b\mathbf{c}_2 + d\mathbf{c}_3 \\[0.4em] &\quad = (a + d)\mathbf{c}_1 + (b + d)\mathbf{c}_2, \end{aligned}

so the span is still the same plane. Three vectors in $\mathbb{R}^3$ , none a multiple of another, can therefore fail to span $\mathbb{R}^3$ . A sharper case uses only the two generators $\mathbf{c}_1$ and $2\mathbf{c}_1$ . Then $a\mathbf{c}_1 + b(2\mathbf{c}_1) = (a + 2b)\mathbf{c}_1$ , so the span is only the line through the origin in the direction of $\mathbf{c}_1$ . The choices $a = 3, b = 0$ and $a = 1, b = 1$ both give $3\mathbf{c}_1 = (3, 0, 3)$ , and the target $(3, 0, 2)$ is out of reach, because every multiple of $\mathbf{c}_1$ has equal first and third entries.

Check yourself

Write $(3, 4)$ as a linear combination of $(1, 2)$ and $(0, 1)$ .

Show the answer

We need $a(1, 2) + b(0, 1) = (a, 2a + b)$ to equal $(3, 4)$ . The first entry gives $a = 3$ , and then $6 + b = 4$ gives $b = -2$ . As a check,

3\begin{bmatrix} 1 \\ 2 \end{bmatrix} - 2\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 6 \end{bmatrix} + \begin{bmatrix} 0 \\ -2 \end{bmatrix} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}.

Is $(4, -1, 2)$ in the span of $\mathbf{c}_1 = (1, 0, 1)$ and $\mathbf{c}_2 = (0, 1, 1)$ ? If not, which coefficient on $\mathbf{e}_3 = (0, 0, 1)$ reaches it once $\mathbf{e}_3$ is added as a third generator?

Show the answer

Every combination $a\mathbf{c}_1 + b\mathbf{c}_2$ equals $(a, b, a + b)$ . The first two entries force $a = 4$ and $b = -1$ , and then the third entry is $3$ , not $2$ . So $(4, -1, 2)$ is not in the span. Checking only the first two entries would wrongly suggest that it is. Once $\mathbf{e}_3$ is added with a coefficient $c$ , every combination equals $(a, b, a + b + c)$ . The first two entries still force $a = 4$ and $b = -1$ , so the third entry is $3 + c$ , and it equals $2$ only for $c = -1$ . Indeed $4\mathbf{c}_1 - \mathbf{c}_2 - \mathbf{e}_3 = (4, -1, 2)$ .

The vectors $(1, 1, 0)$ , $(0, 1, 1)$ and $(1, 2, 1)$ lie in $\mathbb{R}^3$ , and none of them is a multiple of another. Do they span $\mathbb{R}^3$ ?

Show the answer

No. The third vector is the sum of the first two, so it is a redundant generator, and the span equals the span of $(1, 1, 0)$ and $(0, 1, 1)$ . Those combinations are

a\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} + b\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} a \\ a + b \\ b \end{bmatrix},

so the span is the plane $y = x + z$ . For example, $(1, 0, 0)$ is not in it. Its first and third entries would force $a = 1$ and $b = 0$ , and then the middle entry would be $1$ , not $0$ .

Linear Combinations and Span

Key terms

Linear combination

Span

Generating vectors

Membership in a span

Redundant generator

Check yourself

How this lesson connects

Builds on

Used later in