Linear Algebra I reference sheet

This sheet follows the eleven units of the course in order. Each unit starts with the big idea in a few sentences, then gives the definitions, formulas, theorems and algorithms you will look up most often, with the conditions they need. When you want the full explanation, a proof or more examples, follow the lesson links under each topic.

Scalars are real numbers until complex numbers enter in the unit on eigenvalues. Vectors are columns and get bold lowercase letters such as $\mathbf{v}$, with the zero vector $\mathbf{0}$ and the standard unit vectors $\mathbf{e}_1, \dots, \mathbf{e}_n$. In running text a tuple such as $(3, -1)$ stands for the column with those entries. Vectors of an abstract vector space are bold too, while a polynomial $p$ or a matrix used as a vector keeps its usual name. Scalars are plain italic letters, matrices are capitals such as $A$, and a real $m \times n$ matrix is a map from $\mathbb{R}^n$ to $\mathbb{R}^m$. The transpose is $A^{\top}$, the dot product is $\mathbf{x} \cdot \mathbf{y} = \mathbf{x}^{\top}\mathbf{y}$, the norm is $\lVert \mathbf{x} \rVert$ and a general inner product is $\langle \mathbf{u}, \mathbf{v} \rangle$.

A matrix has the subspaces $\operatorname{Col} A$, $\operatorname{Row} A$ and $\operatorname{Null} A$, and a linear map $T$ has $\ker T$ and $\operatorname{im} T$. An ordered basis is $\mathcal{B} = (\mathbf{b}_1, \dots, \mathbf{b}_n)$, the coordinate column of $\mathbf{v}$ in it is $[\mathbf{v}]_{\mathcal{B}}$, and coordinates change by $[\mathbf{v}]_{\mathcal{C}} = P_{\mathcal{C} \leftarrow \mathcal{B}}[\mathbf{v}]_{\mathcal{B}}$. The characteristic polynomial is $\chi_A(t) = \det(tI - A)$, eigenvalues come in no particular order, and singular values are nonnegative and listed in decreasing order. For complex matrices $A^{*} = \overline{A}^{\top}$ and $\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^{*}\mathbf{y}$, which is conjugate-linear in the first argument. Row operations are written as $R_2 \to R_2 - 3R_1$. A formula holds wherever both sides are defined unless a condition says otherwise.

Vectors and Matrices

Vectors are lists of numbers that can be added and scaled, and matrices collect vectors as columns. Multiplying a matrix by a vector forms a weighted sum of its columns. Linear transformations are the maps that respect addition and scaling, and composing two of them multiplies their matrices in a definite order.

Vectors and linear combinations

A vector in $\mathbb{R}^n$ is an ordered list of $n$ real numbers. Vectors of the same size are added and scaled entry by entry,

In the plane and in space a sum follows the tip-to-tail rule, and for $\mathbf{v} \ne \mathbf{0}$ the multiple $c\mathbf{v}$ points along $\mathbf{v}$ when $c > 0$ and against it when $c < 0$, with $\lvert c \rvert$ times the length. A point $P$ is a location, while the displacement from $P$ to $Q$ is the vector $Q - P$, endpoint minus starting point, wherever its arrow is drawn. A data vector lists measurements in a fixed order, and a sum of two data vectors means something only when matching positions hold the same quantity in the same unit.

A linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_k$ is $a_1\mathbf{v}_1 + \cdots + a_k\mathbf{v}_k$ with real coefficients, and $\operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\}$ is the set of all of them. It always contains $\mathbf{0}$. A vector $\mathbf{b}$ lies in the span exactly when the system $x_1\mathbf{v}_1 + \cdots + x_k\mathbf{v}_k = \mathbf{b}$ has a solution. A generator that is a combination of the others is redundant, and removing it leaves the span unchanged.

For example $a(1, 0, 1) + b(0, 1, 1) = (a, b, a + b)$, so these two vectors span the plane $z = x + y$ in $\mathbb{R}^3$. The vector $(2, -1, 1)$ equals $2(1, 0, 1) - (0, 1, 1)$, while $(2, -1, 0)$ is not in the span, because its third entry is not the sum of the first two.

Lessons Vectors and Vector Operations, Linear Combinations and Span

Matrices and the matrix-vector product

An $m \times n$ matrix $A = [a_{ij}]$ has $m$ rows and $n$ columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ in $\mathbb{R}^m$, and the entry $a_{ij}$ sits in row $i$ and column $j$. For $\mathbf{x}$ in $\mathbb{R}^n$,

a vector in $\mathbb{R}^m$ whose entry $i$ is row $i$ times $\mathbf{x}$, that is $a_{i1}x_1 + \cdots + a_{in}x_n$. The product needs exactly $n$ entries in $\mathbf{x}$.

Transposing twice gives $(A^{\top})^{\top} = A$, and $(A + B)^{\top} = A^{\top} + B^{\top}$. The transpose only rearranges entries, and in general it is not the inverse of $A$. The $2 \times 3$ matrix

returns several times on this sheet, always under the name $A$. It maps $\mathbb{R}^3$ to $\mathbb{R}^2$, and

Lessons Matrices and Matrix-Vector Multiplication

Linear transformations

A map $T$ from $\mathbb{R}^n$ to $\mathbb{R}^m$ is linear when, for all $\mathbf{u}$ and $\mathbf{v}$ in $\mathbb{R}^n$ and every real $c$,

These are additivity and homogeneity, and together they say that $T$ keeps every linear combination, $T(a\mathbf{u} + b\mathbf{v}) = aT(\mathbf{u}) + bT(\mathbf{v})$. Every linear map has $T(\mathbf{0}) = \mathbf{0}$, but a map that fixes $\mathbf{0}$ need not be linear.

Standard matrix. A linear $T$ is multiplication by exactly one $m \times n$ matrix, the one whose columns are the images of the standard unit vectors,

Conversely every map $\mathbf{x} \mapsto A\mathbf{x}$ is linear. Two linear maps that agree on $\mathbf{e}_1, \dots, \mathbf{e}_n$ agree everywhere.

The shear $H(x, y) = (x + y, y)$ has $H(\mathbf{e}_1) = (1, 0)$ and $H(\mathbf{e}_2) = (1, 1)$, which are the columns of its matrix. A translation $\mathbf{x} \mapsto \mathbf{x} + \mathbf{b}$ with $\mathbf{b} \ne \mathbf{0}$ is not linear, since it moves $\mathbf{0}$. The map $G(x, y) = (x^2, y)$ fixes $\mathbf{0}$ and is still not linear, because $G(2, 0) = (4, 0)$ while $2G(1, 0) = (2, 0)$. One failed identity proves that a map is not linear, and no number of successful spot checks proves that it is.

Lessons Linear Transformations

Matrix multiplication and composition

If $A$ is $m \times n$ and $B$ is $n \times p$ with columns $\mathbf{b}_1, \dots, \mathbf{b}_p$, then $AB$ is the $m \times p$ matrix

It is defined only when $A$ has as many columns as $B$ has rows. It is the matrix of the composition that applies $B$ first and then $A$, so $(AB)\mathbf{x} = A(B\mathbf{x})$.

Block multiplication. When the column groups of $A$ match the row groups of $B$, the product can be formed block by block, with block $(i, j)$ of $AB$ equal to $\sum_k A_{ik}B_{kj}$. Each block product keeps its order.

For the shear matrix $H$ above and $D = \operatorname{diag}(2, 1)$,

so shearing first and then stretching sends $\mathbf{e}_2$ to $(2, 1)$, while stretching first sends it to $(1, 1)$.

Lessons Matrix Multiplication and Composition

Systems of Linear Equations

A linear system asks for every input that satisfies several linear equations at once. Row operations simplify a system without changing its solutions, and pivots and free variables then describe all of them. A square matrix with a pivot in every column has an inverse, and the LU factorization stores elimination so that it can be reused for new right-hand sides.

Linear systems and their two pictures

A system of $m$ linear equations in $n$ unknowns is one matrix equation $A\mathbf{x} = \mathbf{b}$, with an $m \times n$ coefficient matrix $A$, unknowns $\mathbf{x}$ in $\mathbb{R}^n$ and right-hand side $\mathbf{b}$ in $\mathbb{R}^m$. Its augmented matrix is $[\,A \mid \mathbf{b}\,]$.

• Row picture. Each equation describes the set of points of $\mathbb{R}^n$ that satisfy it, which is a line, plane or hyperplane when its coefficients are not all zero. The solution set is the intersection of all these sets.
• Column picture. The system asks for weights with $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$, so it is consistent exactly when $\mathbf{b}$ lies in the span of the columns.

Every linear system has no solution, exactly one, or infinitely many, because if $\mathbf{p} \ne \mathbf{q}$ are solutions, so is every $\mathbf{p} + t(\mathbf{q} - \mathbf{p})$. A square coefficient matrix does not by itself guarantee one solution, and more equations than unknowns do not rule out infinitely many.

For example $x + y = 3$ and $x - y = 1$ meet at $(2, 1)$, and in the column picture $2(1, 1) + 1(1, -1) = (3, 1)$. The pair $x + y = 2$ and $2x + 2y = 4$ has the whole line of solutions $(t, 2 - t)$, and the pair $x + y = 2$ and $2x + 2y = 5$ has none.

Lessons Linear Systems as Equations and Geometry

Row operations and Gaussian elimination

Each operation acts on whole rows of $[\,A \mid \mathbf{b}\,]$, constants included, and can be undone, so row equivalent augmented matrices have the same solution set. Applying an operation to $I_m$ gives an elementary matrix $E$, and $EA$ performs that operation on $A$. Operations $E_1, \dots, E_k$ applied in that order give $E_k \cdots E_1 A$.

A matrix is in row echelon form when its zero rows are at the bottom, each leading entry lies strictly to the right of the leading entry above it, and every entry below a leading entry is zero. Its leading entries are the pivots. Gaussian elimination reaches an echelon form column by column, swapping in a nonzero pivot when needed and clearing the entries below it. Back-substitution then solves from the last equation upward.

For $x + y + z = 6$, $2x + 3y + z = 11$ and $x + 2y + 3z = 14$, the augmented matrix

becomes, after $R_2 \to R_2 - 2R_1$ and $R_3 \to R_3 - R_1$ and then $R_3 \to R_3 - R_2$,

Back-substitution gives $z = 3$, then $y = -1 + z = 2$, then $x = 6 - y - z = 1$, and $(1, 2, 3)$ satisfies all three original equations.

Lessons Gaussian Elimination and Row Operations

Pivots, free variables and all solutions

The pivot columns of $A$ are the columns that hold pivots in an echelon form, and the unknowns of the other columns are free variables. The reduced row echelon form also has every pivot equal to $1$ and zeros above each pivot, and every matrix has exactly one.

A consistent system with $n$ unknowns and $r$ pivots has $k = n - r$ free variables. Every solution is one particular solution $\mathbf{x}_p$ plus a solution of the homogeneous system $A\mathbf{x} = \mathbf{0}$. Setting each free variable equal to its own parameter and solving for the pivot variables gives the complete solution family

with one real parameter $t_i$ for each free variable and fixed vectors $\mathbf{v}_i$ that satisfy $A\mathbf{v}_i = \mathbf{0}$. For the $2 \times 3$ matrix $A$ and $\mathbf{b} = (4, 3)$, the third unknown is free, and setting $x_3 = t$ gives

Coefficient parameters. A letter among the coefficients selects a system, while a free variable selects a solution within one system. Dividing by an expression in the parameter is valid only where it is nonzero, so each value where it vanishes is solved again from the equations before the division. For

the replacement $R_2 \to R_2 - R_1$ leaves $(a - 1)x_2 = 1$. For $a \ne 1$ the unique solution is

For $a = 1$ the row reads $0 = 1$ and there is no solution. Check every answer in the original equations, with the parameter kept as a letter.

Lessons Pivots, Free Variables and All Solutions, Linear Systems with Parameters, Linear Systems Workshop

Inverse matrices

An $n \times n$ matrix $A$ is invertible when some $n \times n$ matrix $A^{-1}$ satisfies $A^{-1}A = I$ and $AA^{-1} = I$. The inverse is unique, and a square matrix without one is singular. For an $n \times n$ matrix $A$ the following statements are equivalent.

• The matrix $A$ has $n$ pivots.
• The system $A\mathbf{x} = \mathbf{b}$ has exactly one solution for every $\mathbf{b}$ in $\mathbb{R}^n$, namely $\mathbf{x} = A^{-1}\mathbf{b}$.
• The only solution of $A\mathbf{x} = \mathbf{0}$ is $\mathbf{x} = \mathbf{0}$.
• The matrix $A$ is invertible.

For a $2 \times 2$ matrix with $ad - bc \ne 0$,

Computing an inverse. Row reduce $[\,A \mid I\,]$. If the left block becomes $I$, the right block is $A^{-1}$, and if an echelon form of $A$ has fewer than $n$ pivots, then $A$ is singular. To solve a single system, eliminating on $[\,A \mid \mathbf{b}\,]$ takes less work than forming $A^{-1}$.

For example

and this inverse sends $(5, 11)$ to $(2, 1)$, the solution of the system with right-hand side $(5, 11)$.

Lessons Inverse Matrices and Invertibility

LU factorization

Suppose Gaussian elimination reduces a square matrix $A$ to an upper triangular $U$ without row swaps, working through the columns from left to right and applying $R_i \to R_i - \ell_{ij}R_j$ to each row $i > j$ in column $j$. Then

where $L$ is unit lower triangular, with ones on the diagonal and the multiplier $\ell_{ij}$ in position $(i, j)$. To solve $A\mathbf{x} = \mathbf{b}$, solve $L\mathbf{w} = \mathbf{b}$ by forward substitution and then $U\mathbf{x} = \mathbf{w}$ by back-substitution. The factors are computed once and reused for every new right-hand side.

When a zero or small pivot calls for row swaps, the factorization reads $PA = LU$ with a permutation matrix $P$, the identity with its rows reordered, which satisfies $P^{-1} = P^{\top}$. Then solve $L\mathbf{w} = P\mathbf{b}$ and $U\mathbf{x} = \mathbf{w}$. The swaps reorder equations, never unknowns, and a swap made after some multipliers are stored moves those multipliers with their rows. Partial pivoting takes as pivot an entry of largest absolute value in the current column, on or below the current row, so every multiplier has absolute value at most $1$. A zero in a pivot position does not by itself make $A$ singular.

For example

with the multiplier $\ell_{21} = 2$. For $\mathbf{b} = (5, 11)$, forward substitution gives $\mathbf{w} = (5, 1)$ and back-substitution gives $\mathbf{x} = (2, 1)$, and the same factors solve the system with $\mathbf{b} = (1, 1)$, giving $(1, -1)$.

Lessons LU Factorization and Pivoting

Vector Spaces

A subspace is a set of vectors that contains $\mathbf{0}$ and is closed under addition and scaling. Linear independence makes coefficients unique, a basis is an independent list that spans, and the dimension counts the vectors in any basis. Every matrix has four fundamental subspaces, and elimination finds a basis of each.

Subspaces

A subset $W$ of $\mathbb{R}^n$ is a subspace when

• it contains $\mathbf{0}$,
• the sum $\mathbf{u} + \mathbf{v}$ lies in $W$ for all $\mathbf{u}$ and $\mathbf{v}$ in $W$, and
• the multiple $c\mathbf{u}$ lies in $W$ for every $\mathbf{u}$ in $W$ and every real $c$.

A nonempty subset that is closed under both operations already contains $\mathbf{0} = 0\mathbf{u}$. To prove that a set is a subspace, check these conditions for arbitrary members, and to prove that it is not, give one failure.

A translate $\mathbf{p} + W$ of a subspace is an affine set, and it is a subspace only when $\mathbf{p}$ lies in $W$. The solution set of a consistent system $A\mathbf{x} = \mathbf{b}$ is the affine set $\mathbf{x}_p + \{\mathbf{x} \mid A\mathbf{x} = \mathbf{0}\}$.

The plane $z = x + y$ is a subspace of $\mathbb{R}^3$, and the parallel plane $z = x + y + 1$ is not, since it misses $\mathbf{0}$. The union of the two coordinate axes of $\mathbb{R}^2$ is not a subspace, because $(1, 0) + (0, 1)$ leaves it. For $U = \operatorname{span}\{\mathbf{e}_1, \mathbf{e}_2\}$ and $V = \operatorname{span}\{\mathbf{e}_1, \mathbf{e}_3\}$ in $\mathbb{R}^3$, the intersection is $\operatorname{span}\{\mathbf{e}_1\}$ and the sum is $\mathbb{R}^3$.

Lessons Subspaces, Sums and Intersections

Linear independence

A list $\mathbf{v}_1, \dots, \mathbf{v}_k$ is linearly independent when

and dependent otherwise. A combination with coefficients not all zero that equals $\mathbf{0}$ is a dependence relation.

• An independent list gives unique coefficients. Every vector of its span is a combination of the list in exactly one way.
• A list of two or more vectors is dependent exactly when one of its vectors is a combination of the others.
• A list that contains $\mathbf{0}$ is dependent, and two vectors are dependent exactly when one is a multiple of the other.
• More than $n$ vectors in $\mathbb{R}^n$ are always dependent.

Test by elimination. Put the vectors in the columns of a matrix and row reduce. They are independent exactly when every column has a pivot. A column without a pivot gives a free variable, and setting it to $1$ produces a dependence relation.

The vectors $(1, 0, 1)$, $(0, 1, 1)$ and $(1, 1, 2)$ are dependent, with the relation $(1, 0, 1) + (0, 1, 1) - (1, 1, 2) = \mathbf{0}$, even though no two of them are parallel.

Lessons Linear Independence

Bases and dimension

A basis of a subspace $W$ is a list of vectors in $W$ that is linearly independent and spans $W$, so every vector of $W$ is a combination of the list in exactly one way. For an ordered basis $\mathcal{B} = (\mathbf{b}_1, \dots, \mathbf{b}_k)$, the coordinate vector $[\mathbf{v}]_{\mathcal{B}} = (c_1, \dots, c_k)$ lists the coefficients in $\mathbf{v} = c_1\mathbf{b}_1 + \cdots + c_k\mathbf{b}_k$. It has one entry for each basis vector, and reordering the basis reorders the coordinates. The standard basis $(\mathbf{e}_1, \dots, \mathbf{e}_n)$ of $\mathbb{R}^n$ gives each vector its own entries as coordinates.

• Dimension. All bases of $W$ have the same number of vectors, $\dim W$. In $W$ an independent list has at most $\dim W$ vectors and a spanning list has at least $\dim W$. The zero subspace has dimension $0$, and a subspace of $\mathbb{R}^n$ has $\dim W \le n$, with equality only for $W = \mathbb{R}^n$.
• Pruning and extending. Removing, one at a time, a vector that is a combination of the remaining ones shrinks a spanning list to a basis. Adding, one at a time, a vector that lies outside the current span grows an independent list to a basis.
• Counting shortcut. A list of exactly $\dim W$ vectors in $W$ is a basis as soon as it is independent, and also as soon as it spans.

With $\mathbf{b}_1 = (1, 0)$ and $\mathbf{b}_2 = (1, 1)$, every $(x, y)$ equals $(x - y)\mathbf{b}_1 + y\mathbf{b}_2$, so $(3, 2)$ has coordinates $(1, 2)$ and $(4, 1)$ has coordinates $(3, 1)$. The plane $z = x + y$ has dimension $2$, with basis $(1, 0, 1)$, $(0, 1, 1)$, and adding $\mathbf{e}_3$, which lies off the plane, extends it to a basis of $\mathbb{R}^3$.

Lessons Bases and Dimension

The four fundamental subspaces

Let $A$ be a real $m \times n$ matrix of rank $r$, the number of pivots in an echelon form. Its column space $\operatorname{Col} A$ is the span of the columns, which is the set of all $A\mathbf{x}$. Its row space $\operatorname{Row} A = \operatorname{Col} A^{\top}$ is the span of the rows, read as vectors of $\mathbb{R}^n$. Its null space $\operatorname{Null} A$ is the solution set of $A\mathbf{x} = \mathbf{0}$, and its left null space $\operatorname{Null} A^{\top}$ is the solution set of $A^{\top}\mathbf{y} = \mathbf{0}$.

Row operations keep the row space and the null space but usually change the column space, so a basis of $\operatorname{Col} A$ uses the pivot columns of the original matrix. The rank satisfies $\operatorname{rank} A = \dim \operatorname{Col} A = \dim \operatorname{Row} A$, so $\operatorname{rank} A^{\top} = \operatorname{rank} A \le \min(m, n)$. The system $A\mathbf{x} = \mathbf{b}$ is consistent exactly when $\mathbf{b}$ lies in $\operatorname{Col} A$. A matrix of rank $r \ge 1$ has a rank factorization $A = CR$, with $C$ the $m \times r$ matrix of pivot columns and $R$ the $r$ nonzero rows of the reduced row echelon form.

The matrix

has reduced row echelon form with nonzero rows $(1, 2, 0)$ and $(0, 0, 1)$, so its rank is $2$ and columns $1$ and $3$ are the pivot columns.

• The column space has basis $(1, 2, 3)$, $(1, 0, 1)$, taken from $M$ itself.
• The row space has basis $(1, 2, 0)$, $(0, 0, 1)$.
• The null space is spanned by $(-2, 1, 0)$.
• The left null space is spanned by $(-1, -1, 1)$.

Lessons Column Space, Row Space and Rank, Null Space and Left Null Space

Linear Transformations and Coordinates

This unit studies linear maps apart from any one matrix. When the input space is finite-dimensional, the dimensions of the kernel and the image add up to its dimension. A fixed vector gets new coordinates in a new basis, a fixed map gets a new matrix, and polynomials and matrices become vectors with coordinates of their own.

Kernel, image and rank-nullity

For a linear map $T \colon V \to W$, the kernel $\ker T$ is the set of $\mathbf{v}$ in $V$ with $T(\mathbf{v}) = \mathbf{0}$, a subspace of $V$, and the image $\operatorname{im} T$ is the set of all outputs $T(\mathbf{v})$, a subspace of $W$. For $T(\mathbf{x}) = A\mathbf{x}$ they are $\operatorname{Null} A$ and $\operatorname{Col} A$. When $V$ is finite-dimensional, the rank-nullity theorem says

where $\operatorname{rank} T = \dim \operatorname{im} T$. For an $m \times n$ matrix this reads $\dim \operatorname{Null} A + \operatorname{rank} A = n$.

The table assumes $V$ and $W$ finite-dimensional. The fiber over $\mathbf{b}$, the set of inputs with $T(\mathbf{x}) = \mathbf{b}$, is either empty or a translated kernel $\mathbf{x}_p + \ker T$. For linear maps $T \colon U \to V$ and $S \colon V \to W$, the composition satisfies $\ker T \subseteq \ker(S \circ T)$ and $\operatorname{rank}(S \circ T) \le \min(\operatorname{rank} S, \operatorname{rank} T)$, so a later map cannot restore a direction that an earlier one lost.

The $2 \times 3$ matrix $A$ has rank $2$ and nullity $1$, and $2 + 1 = 3$. No linear map from $\mathbb{R}^4$ to $\mathbb{R}^2$ is injective, since its rank is at most $2$ and so its nullity is at least $2$.

Lessons Kernel, Image and Rank-Nullity

Change of basis

Let $\mathcal{B} = (\mathbf{b}_1, \dots, \mathbf{b}_n)$ be an ordered basis of $\mathbb{R}^n$ with basis matrix $B = [\,\mathbf{b}_1 \ \cdots \ \mathbf{b}_n\,]$, which is invertible. Then

and in practice $[\mathbf{v}]_{\mathcal{B}}$ is found by solving $B\mathbf{c} = \mathbf{v}$. For a second ordered basis $\mathcal{C}$ with basis matrix $C$, the change-of-basis matrix from $\mathcal{B}$ to $\mathcal{C}$ is

Its column $j$ is $[\mathbf{b}_j]_{\mathcal{C}}$, its inverse is $P_{\mathcal{B} \leftarrow \mathcal{C}} = B^{-1}C$, and conversions chain as $P_{\mathcal{D} \leftarrow \mathcal{C}}P_{\mathcal{C} \leftarrow \mathcal{B}} = P_{\mathcal{D} \leftarrow \mathcal{B}}$. A change of coordinates is passive. The vector stays where it is and only its description changes, while applying a matrix as a map is active and can move the vector. A coordinate list means something only together with the name of its basis.

Take $\mathcal{B} = ((1, 0), (1, 1))$ and $\mathcal{C} = ((1, 1), (1, -1))$. Then

and the vector $\mathbf{v} = (3, 2)$, with $[\mathbf{v}]_{\mathcal{B}} = (1, 2)$, has $[\mathbf{v}]_{\mathcal{C}} = (\tfrac{5}{2}, \tfrac{1}{2})$. Rebuilding gives $\tfrac{5}{2}(1, 1) + \tfrac{1}{2}(1, -1) = (3, 2)$.

Lessons Change of Basis for Vector Coordinates

Matrices of a linear map

For a linear map $T \colon V \to W$ between finite-dimensional spaces, with an ordered basis $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ of $V$ and an ordered basis $\mathcal{C}$ of $W$, the matrix of $T$ has the output coordinates of the input basis vectors as columns,

It is the only matrix with $[T(\mathbf{v})]_{\mathcal{C}} = [T]_{\mathcal{C} \leftarrow \mathcal{B}}[\mathbf{v}]_{\mathcal{B}}$ for every $\mathbf{v}$ in $V$. For a map from $\mathbb{R}^n$ to $\mathbb{R}^m$ with standard matrix $A$ and basis matrices $B$ and $C$,

A linear operator maps a space to itself and uses one basis on both sides, so $[T]_{\mathcal{B}} = B^{-1}AB$, or equivalently $B[T]_{\mathcal{B}} = AB$, which says that the two routes around the commutative diagram agree.

Similar matrices. Square matrices $A$ and $A'$ are similar when $A' = P^{-1}AP$ for an invertible $P$. The matrices of one operator in two bases are similar, and similar matrices share rank, determinant, trace, characteristic polynomial and eigenvalues. Passing from $A$ to $C^{-1}AB$, with unrelated input and output bases, is more general and is not called similarity. Changing the basis can change the entries, but never the operator, its kernel or its image.

For $A = \operatorname{diag}(2, 1)$ and the basis $\mathcal{B} = ((1, 0), (1, 1))$,

The vector $(3, 2)$ has $\mathcal{B}$-coordinates $(1, 2)$, this matrix sends them to $(4, 2)$, and $4(1, 0) + 2(1, 1) = (6, 2)$ is exactly $A$ times $(3, 2)$.

Lessons Matrices in Different Bases

Vector spaces beyond arrows

A real vector space is a set $V$ with an addition and a multiplication by real scalars that stay in $V$ and obey the usual laws. Addition is commutative and associative, there is a zero vector, every vector has an additive inverse, $a(b\mathbf{u}) = (ab)\mathbf{u}$ and $1\mathbf{u} = \mathbf{u}$, and both distributive laws hold. Subspaces, spans, independence, bases, dimension, kernels and images carry over unchanged.

Here $E_{ij}$ has a $1$ in row $i$ and column $j$ and zeros elsewhere. Coordinates in these bases are the coefficients or the entries, so in the basis $\mathcal{S} = (1, t, t^2)$ the polynomial $2 + 3t - t^2$ has coordinates $(2, 3, -1)$. Polynomials of degree exactly $2$ do not form a vector space, since the zero polynomial is missing and a sum can drop the degree. The degree of a polynomial, the shape of its graph and the dimension of the space it lives in are three different things.

Lessons Vector Spaces of Polynomials and Matrices

Linear maps on polynomials and matrices

A linear map on a finite-dimensional space is determined by the images of a basis, and any choice of images in the target space, repeats allowed, gives exactly one linear map. Its matrix in chosen bases is built column by column as above, and kernel, image and rank-nullity are computed through coordinates. To verify a proposed basis of a subspace $H$, check that each vector lies in $H$, that the list is independent and that it spans $H$, or replace the last check by a dimension count when $\dim H$ is known.

Evaluation. For sample points $s_1, \dots, s_k$, the map $E(p) = (p(s_1), \dots, p(s_k))$ from $P_2$ to $\mathbb{R}^k$ is linear. With $E(p) = (p(0), p(1))$, the basis $(1, t, t^2)$ and the standard basis of $\mathbb{R}^2$, its matrix is

so $E(2 + 3t - t^2) = (2, 4)$, the kernel is $\operatorname{span}\{t^2 - t\}$, the image is $\mathbb{R}^2$, and $1 + 2 = 3 = \dim P_2$. For any two distinct sample points $s_1 \ne s_2$ the kernel is spanned by $(t - s_1)(t - s_2)$.

Trace. The trace $\operatorname{tr} X = x_{11} + \cdots + x_{nn}$ is a linear map from $M_{n \times n}$ to $\mathbb{R}$. On $M_{2 \times 2}$, in the basis $(E_{11}, E_{12}, E_{21}, E_{22})$, it has the $1 \times 4$ matrix $\begin{bmatrix} 1 & 0 & 0 & 1 \end{bmatrix}$, rank $1$ and a kernel of dimension $3$ with basis $E_{12}$, $E_{21}$, $E_{11} - E_{22}$.