Matrices and Matrix-Vector Multiplication

Definition Let $m$ and $n$ be positive integers. An $m \times n$ matrix is a rectangular array of real numbers with $m$ rows and $n$ columns. Its size is $m \times n$, read as $m$ by $n$, with the number of rows first. The entry in row $i$ and column $j$ of a matrix $A$ is written $a_{ij}$, again row first, and we write $A = [a_{ij}]$. Column $j$ of $A$ is a vector $\mathbf{a}_j$ in $\mathbb{R}^m$, and listing the columns gives

Two matrices are equal when they have the same size and the same entry in every position. A matrix with as many rows as columns is square. In running text a tuple such as $(2, 1, 0)$ stands for a column vector.

Definition Let $A$ be an $m \times n$ matrix with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$, and let $\mathbf{x} = (x_1, \dots, x_n)$ be a vector in $\mathbb{R}^n$. The matrix-vector product $A\mathbf{x}$ is the weighted sum of the columns of $A$ with the entries of $\mathbf{x}$ as weights,

In other words, $A\mathbf{x}$ is the linear combination of the columns of $A$ whose coefficients are the entries of $\mathbf{x}$. The product is defined only when $\mathbf{x}$ has exactly $n$ entries, one weight for each column. Since every column lies in $\mathbb{R}^m$, so does $A\mathbf{x}$. An $m \times n$ matrix therefore takes inputs in $\mathbb{R}^n$ and gives outputs in $\mathbb{R}^m$, and we say that $A$ maps $\mathbb{R}^n$ to $\mathbb{R}^m$.

Definition For an $m \times n$ matrix $A = [a_{ij}]$ and a vector $\mathbf{x}$ in $\mathbb{R}^n$, entry $i$ of $A\mathbf{x}$ is

for $i = 1, \dots, m$. This follows from the column definition, because entry $i$ of $x_j\mathbf{a}_j$ is $a_{ij}x_j$ and a sum of vectors is added one position at a time. The summation sign adds the terms for $j = 1, \dots, n$. The row index $i$ stays fixed while $j$ runs across row $i$.

Definition Let $A = [a_{ij}]$ and $B = [b_{ij}]$ be matrices of the same size $m \times n$. Their sum $A + B$ is the $m \times n$ matrix with entries

Matrices of different sizes cannot be added. For every $\mathbf{x}$ in $\mathbb{R}^n$ the sum satisfies $(A + B)\mathbf{x} = A\mathbf{x} + B\mathbf{x}$, because in each row the entry $a_{ij} + b_{ij}$ multiplies $x_j$, and the products split and regroup into the row calculations for $A\mathbf{x}$ and $B\mathbf{x}$.

Definition For a scalar $c$ and an $m \times n$ matrix $A = [a_{ij}]$, the scalar multiple $cA$ is the $m \times n$ matrix with entries

The matrix $(-1)A$ is written $-A$, and for matrices of the same size $A - B$ means $A + (-1)B$. For every $\mathbf{x}$ in $\mathbb{R}^n$ the scalar multiple satisfies $(cA)\mathbf{x} = c(A\mathbf{x})$.

Definition The transpose of an $m \times n$ matrix $A = [a_{ij}]$ is the $n \times m$ matrix $A^{\top}$ with entries

for $i = 1, \dots, n$ and $j = 1, \dots, m$. Row $i$ of $A$ becomes column $i$ of $A^{\top}$, so the entry in row $i$ and column $j$ of $A$ moves to row $j$ and column $i$ without changing its value. Transposing twice returns the original matrix, $(A^{\top})^{\top} = A$. The superscript $\top$ names an operation, not a power. The transpose only rearranges entries, and in general it does not undo multiplication by $A$.