Linear Algebra II reference sheet

This sheet follows the ten units of the course in order. Each unit starts with the big idea in a few sentences, then gives the definitions, theorems and algorithms you will look up most often, each with the hypotheses it needs. When you want a proof or more examples, follow the lesson links under each topic.

Every result names its field $\mathbb{F}$, which is $\mathbb{R}$ or $\mathbb{C}$, and all vector spaces are finite-dimensional unless a statement says otherwise. Vectors such as $v, w, x$ are written in plain italic, coordinates are columns, and a matrix $A \in \mathbb{F}^{m \times n}$ represents a linear map $T: V \to W$ from an $n$-dimensional space to an $m$-dimensional one. The linear maps from $V$ to $W$ form the space $\operatorname{Hom}(V, W)$. Coordinates change by $[v]_{\mathcal{C}} = P_{\mathcal{C} \leftarrow \mathcal{B}}[v]_{\mathcal{B}}$, and the matrix $[T]_{\mathcal{D} \leftarrow \mathcal{B}}$ has as its $j$th column the $\mathcal{D}$ coordinates of $T(b_j)$.

The transpose is $A^{\mathsf{T}}$ and the conjugate transpose is $A^{*} = \overline{A}^{\mathsf{T}}$. Inner products are conjugate-linear in the first argument and linear in the second, with the standard inner product $\langle x, y \rangle = x^{*}y$ on $\mathbb{F}^n$ and $\lVert x \rVert = \sqrt{\langle x, x \rangle}$. The algebraic dual is $V^{\vee}$ and the dual map is $T^{\vee}$, while $T^{*}$ always means the adjoint, which needs an inner product. The space $\mathcal{P}_d$ holds the polynomials of degree at most $d$, including the zero polynomial. The characteristic polynomial is $\chi_A(t) = \det(tI - A)$ and the minimal polynomial $\mu_A$ is monic. Singular values are listed in decreasing order $\sigma_1 \ge \cdots \ge \sigma_p \ge 0$ with $p = \min(m, n)$, while the eigenvalues of a Hermitian matrix are listed in increasing order $\lambda_1 \le \cdots \le \lambda_n$ whenever the order matters. A real symmetric matrix counts as Hermitian.

Vector Spaces and Linear Maps

The first unit proves the facts that every later argument relies on. It sets out how to read and write a proof, fixes the field of scalars, shows that all bases of a space have the same size, and builds the space of linear maps, the rank-nullity theorem and direct sums with their projections.

Proofs and quantifiers

A universal claim $\forall x \in S,\ P(x)$ is proved by an argument about an arbitrary member of $S$ and disproved by one counterexample. An existence claim $\exists x \in S,\ P(x)$ is proved by producing one member with the property, or by an argument that one must exist. An implication $P \Rightarrow Q$ is false only when $P$ holds and $Q$ fails. Negation switches each quantifier.

• Converse and contrapositive. The converse of $P \Rightarrow Q$ is $Q \Rightarrow P$, a different statement. The contrapositive $\neg Q \Rightarrow \neg P$ is equivalent to the original. A statement $P \Leftrightarrow Q$ needs a proof of both directions.
• Induction. If the base case $P(k_0)$ holds and $P(k)$ implies $P(k + 1)$ for every integer $k \ge k_0$, then $P(k)$ holds for every integer $k \ge k_0$. Checking several values of $k$ proves nothing about the others.

For example, if $N^2 = 0$, then $(I + N)^k = I + kN$ for every integer $k \ge 0$. The base case is $(I + N)^0 = I$, and the step is

Lessons Proof Strategies in Linear Algebra

Fields, vector spaces and subspaces

A complex number $z = a + bi$ with $a, b \in \mathbb{R}$ has conjugate $\overline{z} = a - bi$ and modulus $\lvert z \rvert = \sqrt{a^2 + b^2} = \sqrt{z\overline{z}}$. Conjugation respects sums and products, $\lvert zw \rvert = \lvert z \rvert \lvert w \rvert$, and for $z \ne 0$

A vector space over $\mathbb{F}$ allows scalars only from $\mathbb{F}$. Its axioms ask that addition is commutative and associative with a zero vector and negatives, that $a(bv) = (ab)v$ and $1v = v$, and that both distributive laws hold. Everything proved from the axioms alone, such as $0v = 0$ and $(-1)v = -v$, holds in every vector space.

• Subspace test. A subset $U$ of $V$ is a subspace exactly when it is nonempty and $au + bv \in U$ for all $u, v \in U$ and $a, b \in \mathbb{F}$. The line $x + y = 1$ in $\mathbb{R}^2$ fails because it misses $0$, and the union of the two axes fails because $(1, 0) + (0, 1)$ lies on neither.
• Real and complex dimension. If $v_1, \dots, v_n$ is a basis of a complex space $V$, then $v_1, iv_1, \dots, v_n, iv_n$ is a basis of $V$ as a real space, so $\dim_{\mathbb{R}} V = 2\dim_{\mathbb{C}} V$. A statement proved for real spaces does not transfer to complex scalars without a check.
• Polynomials. The space $\mathcal{P}_d$ has basis $1, t, \dots, t^d$, so $\dim \mathcal{P}_d = d + 1$. The polynomials of degree exactly $d$ are not a subspace, since the zero polynomial is missing.

For example, the vector $(2 + i, 3 - 2i) \in \mathbb{C}^2$ has two complex coordinates in the standard basis, and over $\mathbb{R}$ it has the four coordinates $(2, 1, 3, -2)$ in the basis $(1, 0), (i, 0), (0, 1), (0, i)$.

Lessons Real and Complex Vector Spaces

Bases and dimension

The exchange lemma says that if $v_1, \dots, v_m$ are linearly independent in $V$ and $w_1, \dots, w_n$ span $V$, then $m \le n$. Its key step replaces some $w_k$ by a vector $v = \sum_j a_j w_j$ with $a_k \ne 0$, which keeps the span. When $v_1, v_2, \dots$ are inserted one at a time, independence always leaves such a nonzero coefficient on a remaining $w_k$. Applied in both directions to two bases it shows that all bases of $V$ have the same length, the dimension $\dim V$, with $\dim \{0\} = 0$.

• Extracting. Every finite spanning list contains a basis. Remove, one at a time, a vector that is a combination of the others.
• Extending. Every independent list in a finite-dimensional space extends to a basis. Append, one at a time, a vector outside the current span.
• Subspaces. If $U$ is a subspace of $V$, then $\dim U \le \dim V$, with equality only when $U = V$.

For example, $u_1 = (1, 0, 1)$ and $u_2 = (0, 1, 1)$ extend to a basis of $\mathbb{R}^3$ by $e_3$, and $(2, 3, 4) = 2u_1 + 3u_2 - e_3$. The three vectors $u_1, u_2, u_1 + u_2$ span only a plane, so three vectors in $\mathbb{R}^3$ need not form a basis.

Lessons Basis Exchange and Dimension

Linear maps and rank-nullity

A linear map is determined by its values on a basis. If $b_1, \dots, b_n$ is a basis of $V$ and $w_1, \dots, w_n \in W$ are arbitrary, exactly one linear $T: V \to W$ has $T(b_j) = w_j$ for every $j$. With pointwise addition and scaling, $(S + T)(x) = S(x) + T(x)$ and $(aT)(x) = aT(x)$, the set $\operatorname{Hom}(V, W)$ is a vector space over $\mathbb{F}$. For bases $\mathcal{B} = (b_1, \dots, b_n)$ of $V$ and $\mathcal{C} = (c_1, \dots, c_m)$ of $W$, the matrix-unit maps with $E_{ij}(b_k) = \delta_{jk}c_i$ form a basis, and the coordinates of $T$ in it are the entries of $[T]_{\mathcal{C} \leftarrow \mathcal{B}}$. Hence

A map has rank one exactly when $T(x) = \varphi(x)\,w$ for a nonzero $w \in W$ and a nonzero linear $\varphi: V \to \mathbb{F}$. In matrices that is a nonzero column times a nonzero row. The kernel $\ker T$ and the image $\operatorname{im} T$ are subspaces, and when $V$ is finite-dimensional the rank-nullity theorem says

When $\dim V = \dim W$ is finite, an injective or a surjective linear map is already an isomorphism. With unequal dimensions this fails, as the inclusion of $\mathbb{R}^2$ in $\mathbb{R}^3$ shows. For composable maps, $\operatorname{rank}(ST) \le \min(\operatorname{rank} S, \operatorname{rank} T)$.

For example, $T(p) = (p(0), p(1))$ from $\mathcal{P}_2$ to $\mathbb{R}^2$ sends $a + bt + ct^2$ to $(a, a + b + c)$. Its kernel is spanned by $t^2 - t$ and it is onto, so $3 = 1 + 2$. All polynomials with measurements $(2, 5)$ are $2 + 3t + s(t^2 - t)$ with $s \in \mathbb{R}$.

Lessons The Vector Space of Linear Maps, Rank-Nullity and Isomorphisms

Sums, direct sums and projections

For subspaces $U$ and $W$ of $V$, the sum $U + W = \{u + w \mid u \in U,\ w \in W\}$ and the intersection $U \cap W$ are subspaces, and for finite-dimensional $U$ and $W$

The union $U \cup W$ is a subspace only when one of them contains the other. The product $U \times W$ of two spaces, with componentwise operations, has dimension $\dim U + \dim W$.

• Direct sum. The space $V$ is the direct sum $V = U \oplus W$ when $U + W = V$ and $U \cap W = \{0\}$, which holds exactly when every $v \in V$ is $u + w$ with $u \in U$ and $w \in W$ in only one way. Then $\dim V = \dim U + \dim W$.
• Complements. In a finite-dimensional space every subspace $U$ has a complement $W$ with $V = U \oplus W$, found by extending a basis of $U$. It is unique only when $U = \{0\}$ or $U = V$.
• Projection along a complement. If $V = U \oplus W$, the map $P(u + w) = u$ is linear with $P^2 = P$, $\operatorname{im} P = U$ and $\ker P = W$, and $I - P$ projects onto $W$ along $U$.
• Idempotents. Conversely, every linear $P$ with $P^2 = P$ gives $V = \operatorname{im} P \oplus \ker P$ and is the projection onto $\operatorname{im} P$ along $\ker P$. The splitting $V = \operatorname{im} P \oplus \ker P$ alone does not force $P^2 = P$, as $P = 2I$ shows.

A projection needs no inner product. With the dot product on $\mathbb{R}^n$, an idempotent real matrix is the orthogonal projection onto its image exactly when it is symmetric, and otherwise it is an oblique projection. For example, the projection onto $U = \operatorname{span}(1, 0)$ along $W = \operatorname{span}(1, 1)$ is $P(x, y) = (x - y, 0)$, with matrix

and it splits $(3, 2)$ as $(1, 0) + (2, 2)$. The matrix is not symmetric, so the projection is oblique.

Lessons Sums and Intersections of Subspaces, Direct Sums and Projections

Quotient Spaces and Duality

A quotient space treats vectors that differ by an element of a fixed subspace as one object, and the dual space collects all linear measurements of vectors. Neither construction needs an inner product. Together they describe what a map cannot detect, explain why measurements change coordinates in the opposite way to vectors, and show why row rank equals column rank.

Quotient spaces

Let $W$ be a subspace of $V$. Vectors $v$ and $u$ of $V$ are equivalent modulo $W$ when $v - u \in W$. The coset of $v$ is

and $v + W = u + W$ exactly when $v - u \in W$. Two cosets are equal or disjoint, and a coset is a subspace only when it is $W$ itself. The quotient space $V/W$ is the set of cosets with the operations

which are well defined because changing representatives changes the results by vectors of $W$. Its zero vector is the coset $W$.

• Well-defined rules. A rule $v + W \mapsto f(v)$ defines a function on $V/W$ exactly when $v - u \in W$ implies $f(v) = f(u)$. For a linear $f$ this means $W \subseteq \ker f$.
• Quotient basis. If $w_1, \dots, w_r$ is a basis of $W$ extended to a basis $w_1, \dots, w_r, v_1, \dots, v_s$ of $V$, then the cosets $v_1 + W, \dots, v_s + W$ form a basis of $V/W$.

When $V$ is finite-dimensional, counting that basis gives

For example, in $\mathbb{R}^3$ with $W = \operatorname{span}(e_3)$, each coset is a vertical line $(a, b, c) + W$. The function $a + b$ is constant on each line and descends to the quotient, while $c$ is not and does not.

Lessons Quotient Spaces

The first isomorphism theorem

Let $T: V \to W$ be linear over $\mathbb{F}$. The fiber over $y$ is $T^{-1}(y) = \{v \mid T(v) = y\}$, which is empty when $y \notin \operatorname{im} T$ and equals the coset $v_0 + \ker T$ when $T(v_0) = y$. The notation does not assume an inverse.

• Quotient map. For a subspace $U$ of $V$, the map $q(v) = v + U$ from $V$ to $V/U$ is linear and onto, with kernel $U$.
• Induced map. The rule $\widetilde{T}(v + U) = T(v)$ defines a map on $V/U$ exactly when $U \subseteq \ker T$. It is then linear, satisfies $T = \widetilde{T} \circ q$, and is the only map with that property.

The first isomorphism theorem says that the induced map

is a linear bijection, so $V/\ker T \cong \operatorname{im} T$. The target is the image, not the codomain, and no finite-dimensional hypothesis is needed. In finite dimensions, counting dimensions recovers rank-nullity.

For example, $T(x, y, z) = (x + y, z)$ on $\mathbb{R}^3$ has $\ker T = \operatorname{span}(1, -1, 0)$, and the coset of $(x, y, z)$ corresponds to $(x + y, z)$. The cosets of $(1, 0, 0)$ and $(0, 0, 1)$ form a quotient basis. Evaluation $\operatorname{ev}_s(p) = p(s)$ maps $\mathcal{P}_d$ onto $\mathbb{F}$, so $\mathcal{P}_d / \ker \operatorname{ev}_s \cong \mathbb{F}$, and for $d \ge 1$ the kernel consists of the multiples $(t - s)r(t)$ with $r \in \mathcal{P}_{d-1}$.

Lessons The First Isomorphism Theorem

Dual spaces and dual bases

A linear functional on $V$ is a linear map $\ell: V \to \mathbb{F}$. Over $\mathbb{C}$ scalars come out with no conjugate. The dual space is

with pointwise operations. Many books write $V^{*}$ for the dual space. This course keeps $V^{\vee}$ so that the dual is not confused with the adjoint $T^{*}$. On $\mathbb{F}^n$ every functional is $\ell(x) = rx$ for exactly one row $r$, whose entries are $\ell(e_1), \dots, \ell(e_n)$, with no conjugation.

• Dual basis. For a basis $b_1, \dots, b_n$ of $V$, the coordinate functionals $\beta_1, \dots, \beta_n$ with $\beta_i(b_j) = \delta_{ij}$ form a basis of $V^{\vee}$, and every functional expands as $\ell = \sum_i \ell(b_i)\,\beta_i$. So $\dim V^{\vee} = \dim V$. On $\mathbb{F}^n$, if the $b_j$ are the columns of $B$, the $\beta_i$ are the rows of $B^{-1}$.
• Evaluation functionals. On $\mathcal{P}_d$ with basis $1, t, \dots, t^d$ and dual basis $\beta_1, \dots, \beta_{d+1}$, evaluation at a fixed $s$ is $\operatorname{ev}_s = \sum_{k=1}^{d+1} s^{k-1}\beta_k$.
• Not functionals. On a complex space, a conjugate-linear map $f$, with $f(av) = \overline{a}f(v)$, is not in $V^{\vee}$ unless it is zero. On a nonzero space the norm $x \mapsto \lVert x \rVert$ is not linear either.

The equality $\dim V^{\vee} = \dim V$ does not single out an isomorphism between the two spaces. Matching $b_i$ with $\beta_i$ depends on the basis. For example, the basis $(1, 1), (1, -1)$ of $\mathbb{R}^2$ has dual basis $\beta_1(x, y) = (x + y)/2$ and $\beta_2(x, y) = (x - y)/2$, and the functional $\varphi(x, y) = 2x - y$ equals $\beta_1 + 3\beta_2$. The vector $(3, 1)$ has coordinates $(2, 1)$ in this basis, and both descriptions give $\varphi(3, 1) = 5$.

Lessons Dual Spaces and Dual Bases

Dual maps and covector coordinates

A covector is another name for a linear functional. The dual map of a linear $T: V \to W$ pulls measurements back,

It is linear over $\mathbb{R}$ and over $\mathbb{C}$, it uses no inner product, it reverses compositions as $(ST)^{\vee} = T^{\vee}S^{\vee}$, and it satisfies $(T^{-1})^{\vee} = (T^{\vee})^{-1}$ for an isomorphism $T$. If $A = [T]_{\mathcal{C} \leftarrow \mathcal{B}}$ and $\mathcal{B}^{\vee}$, $\mathcal{C}^{\vee}$ are the dual bases, then

the plain transpose even over $\mathbb{C}$. In rows, a functional with evaluation row $r$ on $W$ pulls back to the row $rA$ on $V$.

The reading $\ell(v) = r_{\mathcal{C}}[v]_{\mathcal{C}} = r_{\mathcal{B}}[v]_{\mathcal{B}}$ does not depend on the basis, which is why covector coordinates use the inverse. Multiplying both the coefficient column of $\ell$ and the coordinate column of $v$ by $P$ would keep every reading only when $P^{\mathsf{T}}P = I$.

For example, with $A$ having rows $(1, 2)$ and $(0, 1)$ and the output sensor $r = (3, -1)$, the pullback is $rA = (3, 5)$. At $v = (1, 2)$ both routes give $13$, since $Av = (5, 2)$. For $\varphi(x, y) = 2x - y$, standard $\mathcal{B}$ and $\mathcal{C} = ((1, 1), (1, -1))$, the row $r_{\mathcal{B}} = (2, -1)$ becomes $r_{\mathcal{C}} = (1, 3)$, the coefficients of $\varphi$ in the dual basis above.

Lessons Dual Maps and Covectors

Annihilators and the four subspaces

The annihilator of a subspace $U$ of $V$ is

a subspace of $V^{\vee}$, not of $V$. For finite-dimensional $V$, $\dim U^{0} = \dim V - \dim U$. For a linear $T: V \to W$,

where the first identity holds for all vector spaces and the second follows by counting dimensions when $V$ and $W$ are finite-dimensional. Hence $\operatorname{rank} T^{\vee} = \operatorname{rank} T$, and $T^{\vee}$ is injective exactly when $T$ is surjective. For a matrix $A \in \mathbb{F}^{m \times n}$ of rank $r$, the dual map sends an output row $y$ to $yA$, which gives the row space as $(\operatorname{Null} A)^{0}$ and the left nullspace as $(\operatorname{Col} A)^{0}$.

An annihilator needs no extra structure, while an orthogonal complement $U^{\perp}$ lies in $V$ and depends on an inner product. In a finite-dimensional inner product space, the functional $\langle v, \cdot \rangle$ lies in $U^{0}$ exactly when $v \in U^{\perp}$, and the correspondence is conjugate-linear over $\mathbb{C}$. For example, $A$ with rows $(1, 1, 0)$ and $(0, 0, 1)$ has kernel spanned by $(1, -1, 0)$. The rows that vanish on it are $(a, a, b)$, exactly the row space, and since $A$ maps onto $\mathbb{R}^2$ the left nullspace is zero.

Lessons Annihilators and Fundamental Subspaces

Determinants and Bilinear Forms

The determinant is the only function of the columns of a square matrix that is multilinear, alternating and equal to $1$ at the identity, and its product rule follows from that uniqueness. Trace, determinant and characteristic polynomial stay the same under similarity, the rule by which the matrix of an operator changes, while the matrix of a bilinear form changes by congruence. The same square array can therefore follow two different coordinate laws.

The determinant from alternating forms

A function $D(v_1, \dots, v_k)$ of $k$ vectors is multilinear if it is linear in each slot with the others fixed, and alternating if it is $0$ whenever two arguments are equal. An alternating multilinear form changes sign when two arguments are swapped, is unchanged when a multiple of one argument is added to another, and vanishes on linearly dependent arguments. The sign of a permutation $\sigma$ of $\{1, \dots, n\}$ is $\operatorname{sgn}(\sigma) = (-1)^{N(\sigma)}$, where $N(\sigma)$ counts the pairs $i < j$ with $\sigma(i) > \sigma(j)$, and every exchange of two entries flips it. For $A = [a_{ij}] \in \mathbb{F}^{n \times n}$,

The uniqueness theorem says that every function $D$ of the columns that is multilinear and alternating satisfies

so $\det$ is the only one with $D(I) = 1$. Applying it to $D(B) = \det(AB)$ gives $\det(AB) = \det A \det B$ for all square $A$ and $B$ of the same size, singular or not. Also $\det A^{\mathsf{T}} = \det A$, so every column rule holds for rows, and $A$ is invertible exactly when $\det A \ne 0$, with $\det(A^{-1}) = 1/\det A$.

Lessons Determinants and Multilinear Forms

Trace, similarity and the characteristic polynomial

The trace $\operatorname{tr} A$ is the sum of the diagonal entries. For $A \in \mathbb{F}^{m \times n}$ and $B \in \mathbb{F}^{n \times m}$,

even when the two products have different sizes. With three factors only the cyclic rotations $\operatorname{tr}(ABC) = \operatorname{tr}(BCA) = \operatorname{tr}(CAB)$ are allowed. Matrices $A, B \in \mathbb{F}^{n \times n}$ are similar when $B = S^{-1}AS$ for an invertible $S \in \mathbb{F}^{n \times n}$. Similar matrices are exactly the matrices of one operator in two bases. If $S = P_{\mathcal{B} \leftarrow \mathcal{C}}$, then $[T]_{\mathcal{C} \leftarrow \mathcal{C}} = S^{-1}[T]_{\mathcal{B} \leftarrow \mathcal{B}}S$.

The characteristic polynomial of $A \in \mathbb{F}^{n \times n}$ is monic of degree $n$. For $n \ge 2$ it expands as

and for $n = 2$ it is $t^2 - (\operatorname{tr} A)\,t + \det A$. A scalar $\lambda \in \mathbb{F}$ is an eigenvalue of $A$ exactly when $\chi_A(\lambda) = 0$. Trace, determinant, characteristic polynomial, rank and the dimension of every eigenspace are similarity invariants, so they are defined for an operator on a finite-dimensional space. Equal invariants do not imply similarity. The identity and the shear with rows $(1, 1)$ and $(0, 1)$ share trace $2$, determinant $1$ and characteristic polynomial $(t - 1)^2$, but the shear is not similar to $I$, since $S^{-1}IS = I$ for every invertible $S$.

Over $\mathbb{C}$ a determinant is not a signed volume. Multiplication by $A \in \mathbb{C}^{n \times n}$, viewed as a real map on $\mathbb{R}^{2n}$, scales real volume by $\lvert \det A \rvert^2$. For example, $D = \operatorname{diag}(2, 3)$ and $S$ with rows $(1, 1)$ and $(0, 1)$ give

with the same determinant $6$, trace $5$ and characteristic polynomial $t^2 - 5t + 6$ as $D$.

Lessons Trace, Determinant and Similarity

Bilinear forms and congruence

A bilinear form on a real space $V$ is a function $b: V \times V \to \mathbb{R}$ that is linear in each argument. In an ordered basis $\mathcal{B} = (v_1, \dots, v_n)$ its matrix $M_{\mathcal{B}}$ has entries $m_{ij} = b(v_i, v_j)$, and with $\xi = [x]_{\mathcal{B}}$ and $\eta = [y]_{\mathcal{B}}$

The form is symmetric exactly when $M_{\mathcal{B}}$ is, and nondegenerate exactly when $M_{\mathcal{B}}$ is invertible. If the columns of $S = P_{\mathcal{B} \leftarrow \mathcal{C}}$ hold the new basis vectors in old coordinates, then

which is the congruence rule. With $P = P_{\mathcal{C} \leftarrow \mathcal{B}} = S^{-1}$ it reads $M_{\mathcal{C}} = (P^{-1})^{\mathsf{T}} M_{\mathcal{B}}\, P^{-1}$.

• Quadratic form. The function $q(v) = b(v, v)$ satisfies $q(cv) = c^2 q(v)$.

• Polarization. For a symmetric $b$, the polarization identity

  $$b(x, y) = \frac{1}{2}\bigl(q(x + y) - q(x) - q(y)\bigr)$$

  recovers $b$ from $q$, so symmetric forms and quadratic forms correspond one to one.

• Symmetric and skew parts. Every real square $M$ is $H + K$ with $H = \tfrac{1}{2}(M + M^{\mathsf{T}})$ symmetric and $K = \tfrac{1}{2}(M - M^{\mathsf{T}})$ skew-symmetric. Since $x^{\mathsf{T}}Kx = 0$, the quadratic form sees only $H$.

For example, $M_{\mathcal{B}} = \operatorname{diag}(1, 4)$ and the new basis $(2, 0), (0, 1)$ give $S = \operatorname{diag}(2, 1)$ and $M_{\mathcal{C}} = \operatorname{diag}(4, 4)$. The eigenvalues changed, which similarity never does. The vector $(2, 1)$ has $\mathcal{C}$ coordinates $(1, 1)$, and both matrices give $q = 8$. The matrix with rows $(0, 1)$ and $(-1, 0)$ is nonzero but has $q = 0$ everywhere.

Lessons Bilinear Forms and Congruence

Inner Product Spaces

An inner product measures lengths and angles. Over the complex numbers it needs a conjugate, which keeps the squared length of every nonzero vector positive. This unit proves the Cauchy-Schwarz inequality, builds orthonormal bases and expansions, matches every linear functional with a vector, defines the adjoint, and finds best approximations, including weighted least squares, where changing the inner product can move the best fit.

Inner products, norms and Gram matrices

An inner product on a vector space $V$ over $\mathbb{F}$ assigns a scalar $\langle x, y \rangle$ to each pair so that, for all vectors and scalars,

These are linearity in the second argument, conjugate symmetry and positive definiteness. They force $\langle ax, y \rangle = \overline{a}\langle x, y \rangle$, so the first argument is conjugate-linear. The standard inner product on $\mathbb{F}^n$ is $\langle x, y \rangle = x^{*}y = \sum_j \overline{x_j}\,y_j$.

• Norm. The norm $\lVert x \rVert = \sqrt{\langle x, x \rangle}$ has $\lVert ax \rVert = \lvert a \rvert\,\lVert x \rVert$, and $\lVert x + y \rVert^2 = \lVert x \rVert^2 + \lVert y \rVert^2 + 2\operatorname{Re}\langle x, y \rangle$. Orthogonal vectors, with $\langle x, y \rangle = 0$, satisfy the Pythagorean identity.
• Sesquilinear forms. A map that is linear in the second argument and conjugate-linear in the first is sesquilinear, and Hermitian if $s(y, x) = \overline{s(x, y)}$. A complex inner product is exactly a positive definite Hermitian sesquilinear form. On $\mathbb{C}^n$ these forms are $s(x, y) = x^{*}Hy$, Hermitian exactly when $H^{*} = H$.
• Gram matrix. For a basis $b_1, \dots, b_n$ the Gram matrix has entries $G_{ij} = \langle b_i, b_j \rangle$, and with coordinate columns $\xi$ and $\eta$ we have $\langle x, y \rangle = \xi^{*}G\eta$. It is Hermitian and positive definite, it equals $I$ exactly for an orthonormal basis, and a change of basis replaces it by $S^{*}GS$, where the columns of $S$ hold the new basis vectors in old coordinates.

For example, $x = (1, i)$ has $x^{\mathsf{T}}x = 1 + i^2 = 0$ although $x \ne 0$, while $x^{*}x = 2$. Scalars leave the first slot conjugated, so $\langle ix, y \rangle = -i\langle x, y \rangle$ and $\langle x, iy \rangle = i\langle x, y \rangle$.

Lessons Complex Inner Products

Cauchy-Schwarz and orthonormal expansions

For all $x, y$ in an inner product space over $\mathbb{F}$, in any dimension,

with equality exactly when $x$ and $y$ are linearly dependent. For $x \ne 0$ the proof expands $\lVert y - cx \rVert^2 \ge 0$ with $c = \langle x, y \rangle / \lVert x \rVert^2$, which makes $y - cx$ orthogonal to $x$. It gives the triangle inequality $\lVert x + y \rVert \le \lVert x \rVert + \lVert y \rVert$, with equality exactly when one vector is a nonnegative real multiple of the other.

The complex Gram-Schmidt process turns linearly independent $a_1, \dots, a_n$ into an orthonormal list with the same spans $\operatorname{span}(q_1, \dots, q_k) = \operatorname{span}(a_1, \dots, a_k)$ for every $k$. Set $q_1 = a_1/\lVert a_1 \rVert$ and, for $k \ge 2$,

The basis vector sits in the first slot of each coefficient. With dependent inputs, the first $a_k$ in the span of the earlier ones gives $r_k = 0$, or is itself zero when $k = 1$, and that zero signals the dependence. For an orthonormal basis $q_1, \dots, q_n$ of $V$ and every $v \in V$,