Complex Inner Products, linear algebra II lesson

The vector $x = (1, i)$ in $\mathbb{C}^2$ is not zero, yet the real recipe for squared length gives $x^{\mathsf{T}}x = 1 + i^2 = 0$ . A squared length that vanishes on a nonzero vector is useless, so the recipe has to change once the scalars are complex. Conjugating one of the two copies repairs it, because $(a - ib)(a + ib) = a^2 + b^2$ for real $a$ and $b$ . This lesson builds the complex inner product on that observation, settles which argument is linear, and shows how a basis stores an inner product in its Gram matrix. Throughout, a tuple such as $(1, i)$ stands for a column vector.

Key terms

Conjugate transpose

Definition The conjugate transpose of a matrix $A \in \mathbb{C}^{m \times n}$ is the matrix $A^{*} \in \mathbb{C}^{n \times m}$ obtained by transposing and conjugating every entry,

A^{*} = \overline{A}^{\mathsf{T}}, \qquad (A^{*})_{ij} = \overline{a_{ji}}.

A column $x \in \mathbb{C}^n$ becomes the row $x^{*} = \begin{bmatrix} \overline{x_1} & \cdots & \overline{x_n} \end{bmatrix}$ , so for columns $x, y \in \mathbb{C}^n$

x^{*}y = \overline{x_1}\,y_1 + \cdots + \overline{x_n}\,y_n.

The rules $(A^{*})^{*} = A$ , $(cA)^{*} = \overline{c}\,A^{*}$ and $(AB)^{*} = B^{*}A^{*}$ hold whenever the sizes allow the product. For a real matrix the conjugate transpose is the transpose. A square matrix $H$ with $H^{*} = H$ is called Hermitian.

In plain words Turn rows into columns and conjugate every entry. In the product $x^{*}x$ each coordinate meets its own conjugate, and $\overline{x_j}\,x_j = \lvert x_j \rvert^2$ is never negative, while the unconjugated $x_j^2$ can be negative or not even real.

Example For $x = (1, i)$ the two recipes disagree,

\begin{gathered} x^{\mathsf{T}}x = 1 + i^2 = 0, \\[0.4em] x^{*}x = 1 + (-i)(i) = 2. \end{gathered}

For a rectangular matrix,

\begin{gathered} A = \begin{bmatrix} 1 & 2 + i & -i \\ 3 & 0 & 1 + i \end{bmatrix}, \\[0.4em] A^{*} = \begin{bmatrix} 1 & 3 \\ 2 - i & 0 \\ i & 1 - i \end{bmatrix}. \end{gathered}

Complex inner product

Definition Let $V$ be a vector space over $\mathbb{F}$ , where $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$ . An inner product on $V$ assigns to each pair $x, y \in V$ a scalar $\langle x, y \rangle \in \mathbb{F}$ such that for all $x, y, z \in V$ and all $a, b \in \mathbb{F}$

\begin{aligned} &\langle x, ay + bz \rangle \\[0.4em] &\quad = a\langle x, y \rangle + b\langle x, z \rangle, \\[0.4em] &\langle y, x \rangle = \overline{\langle x, y \rangle}, \\[0.4em] &\langle x, x \rangle > 0 \ \text{ whenever } x \ne 0. \end{aligned}

The first rule is linearity in the second argument. The second is conjugate symmetry, which forces every value $\langle x, x \rangle$ to be real. The third is positive definiteness, treated in more detail below. The first argument is then conjugate-linear, since

\begin{aligned} &\langle ax + bz, y \rangle = \overline{\langle y, ax + bz \rangle} \\[0.4em] &\quad = \overline{a\langle y, x \rangle + b\langle y, z \rangle} \\[0.4em] &\quad = \overline{a}\langle x, y \rangle + \overline{b}\langle z, y \rangle. \end{aligned}

When $\mathbb{F} = \mathbb{C}$ this is a complex inner product, and $V$ with it is a complex inner product space. Over $\mathbb{R}$ the conjugations do nothing and the rules are those of a real inner product. The standard inner product on $\mathbb{C}^n$ is $\langle x, y \rangle = x^{*}y$ . Some books make the first argument linear instead. Every formula in this course uses the convention above, with the second argument linear.

In plain words An inner product turns two vectors into a scalar. A scalar factor comes out of the second slot unchanged and out of the first slot conjugated, and swapping the two vectors conjugates the result.

Example The standard inner product satisfies the rules. It is linear in $y$ by matrix multiplication, conjugating $\sum_j \overline{x_j}\,y_j$ gives $\sum_j \overline{y_j}\,x_j = y^{*}x$ , and $x^{*}x = \sum_j \lvert x_j \rvert^2$ is positive unless every entry is $0$ . For $x = (1, i)$ and $y = (1, 1)$ the value is $\langle x, y \rangle = 1 - i$ . Multiplying either input by $i$ gives

\begin{aligned} \langle ix, y \rangle &= \overline{i}\,(1 - i) = -1 - i, \\[0.4em] \langle x, iy \rangle &= i\,(1 - i) = 1 + i, \end{aligned}

and swapping gives $\langle y, x \rangle = 1 + i$ , the conjugate of $1 - i$ .

Sesquilinear form

Definition Let $V$ be a complex vector space. A sesquilinear form on $V$ is a map $s \colon V \times V \to \mathbb{C}$ that is linear in its second argument and conjugate-linear in its first, so that for all $x, y, z \in V$ and $a, b \in \mathbb{C}$

\begin{aligned} s(x, ay + bz) &= a\,s(x, y) \\[0.4em] &\quad + b\,s(x, z), \\[0.4em] s(ax + bz, y) &= \overline{a}\,s(x, y) \\[0.4em] &\quad + \overline{b}\,s(z, y). \end{aligned}

It is Hermitian if $s(y, x) = \overline{s(x, y)}$ for all $x, y$ , and then every value $s(x, x)$ is real. A complex inner product is exactly a Hermitian sesquilinear form that is positive definite. On $\mathbb{C}^n$ every sesquilinear form is $s(x, y) = x^{*}Hy$ for exactly one matrix $H \in \mathbb{C}^{n \times n}$ , namely $h_{ij} = s(e_i, e_j)$ , and the form is Hermitian exactly when $H^{*} = H$ . A complex bilinear form, linear in both arguments, is a different object. If the new basis vectors have old coordinate columns forming an invertible matrix $S$ , a sesquilinear form’s matrix changes by conjugate congruence $H \mapsto S^{*}HS$ , while a bilinear form’s matrix changes by congruence $M \mapsto S^{\mathsf{T}}MS$ .

In plain words Sesquilinear means one and a half times linear. One slot is fully linear and the other is linear up to conjugating scalars, and that half step is what lets a complex form give real, positive squared lengths.

Example The form $b(x, y) = x^{\mathsf{T}}y$ on $\mathbb{C}^2$ is bilinear. For $x = (1, 2)$ it gives $b(x, x) = 5$ , while $b(ix, ix) = i^2 \cdot 5 = -5$ . The same computation works for any bilinear form and any $x$ with $b(x, x) > 0$ , so no complex bilinear form on a nonzero complex space is positive definite. The form

s(x, y) = \overline{x_1}\,y_1 - \overline{x_2}\,y_2,

with matrix $\operatorname{diag}(1, -1)$ , is Hermitian sesquilinear, but $s(e_2, e_2) = -1$ . It meets every rule of an inner product except positivity.

Positive definiteness

Definition A Hermitian sesquilinear form $s$ on a complex vector space $V$ is positive definite if $s(x, x) > 0$ for every nonzero $x \in V$ . A Hermitian matrix $H \in \mathbb{C}^{n \times n}$ is positive definite if

\begin{gathered} z^{*}Hz > 0 \\[0.4em] \text{for every nonzero } z \in \mathbb{C}^n. \end{gathered}

The number $z^{*}Hz$ is real because $H$ is Hermitian, so the condition makes sense, and a positive definite $H$ makes $\langle x, y \rangle_H = x^{*}Hy$ an inner product on $\mathbb{C}^n$ . Over $\mathbb{R}$ the same definitions apply to real vectors, and a real Hermitian matrix is simply a symmetric one. Linearity already gives $s(0, 0) = 0$ , so positive definiteness says that squared length is zero only for the zero vector and is never negative. A single nonzero vector with $s(x, x) \le 0$ rules a form out.

In plain words Every nonzero vector must get a positive squared length. Zero is reserved for the zero vector, and negative squared lengths are not allowed at all.

Example Take $H$ with rows $(2, 1)$ and $(1, 2)$ . Because $\lvert z_1 + z_2 \rvert^2 = \lvert z_1 \rvert^2 + \lvert z_2 \rvert^2 + \overline{z_1}z_2 + \overline{z_2}z_1$ ,

\begin{aligned} z^{*}Hz &= 2\lvert z_1 \rvert^2 + 2\lvert z_2 \rvert^2 \\[0.4em] &\quad + \overline{z_1}z_2 + \overline{z_2}z_1 \\[0.4em] &= \lvert z_1 + z_2 \rvert^2 \\[0.4em] &\quad + \lvert z_1 \rvert^2 + \lvert z_2 \rvert^2. \end{aligned}

Every term is nonnegative, and the last two vanish together only when $z = 0$ . So $H$ is positive definite and $x^{*}Hy$ is an inner product on $\mathbb{C}^2$ , with no eigenvalues needed. At $z = (1, i)$ the value is $2 + 1 + 1 = 4$ . The Hermitian matrices $\operatorname{diag}(1, -1)$ and $\operatorname{diag}(1, 0)$ fail, since the nonzero vector $e_2$ gives $-1$ and $0$ .

Norm from an inner product

Definition In an inner product space over $\mathbb{F}$ , the norm of $x$ is the nonnegative square root

\lVert x \rVert = \sqrt{\langle x, x \rangle}.

It satisfies $\lVert x \rVert \ge 0$ , with equality only for $x = 0$ , and $\lVert ax \rVert = \lvert a \rvert\,\lVert x \rVert$ for every scalar $a$ , because $\langle ax, ax \rangle = \overline{a}\,a\,\langle x, x \rangle$ . Vectors $x$ and $y$ are orthogonal when $\langle x, y \rangle = 0$ , and by conjugate symmetry this holds exactly when $\langle y, x \rangle = 0$ . Expanding the sum gives

\begin{aligned} \lVert x + y \rVert^2 &= \lVert x \rVert^2 + \lVert y \rVert^2 \\[0.4em] &\quad + 2\operatorname{Re}\langle x, y \rangle, \end{aligned}

so orthogonal vectors satisfy $\lVert x + y \rVert^2 = \lVert x \rVert^2 + \lVert y \rVert^2$ . The triangle inequality $\lVert x + y \rVert \le \lVert x \rVert + \lVert y \rVert$ , which makes this a norm in the full sense, is not automatic. It follows from the Cauchy-Schwarz inequality in the next lesson.

In plain words Length comes from pairing a vector with itself. Multiplying a vector by a complex number of absolute value $1$ , a phase, can change its coordinates but never its length.

Example For $x = (1, i)$ and $y = (i, 1)$ , both squared norms are $2$ and $\langle x, y \rangle = 1 \cdot i + (-i) \cdot 1 = 0$ , so the vectors are orthogonal. Their sum $(1 + i, 1 + i)$ has squared norm $2 + 2 = 4$ , the sum of the squared norms. For every real $\theta$ the vector $e^{i\theta}x$ still has norm $\sqrt{2}$ , and $(3 + 4i)x$ has norm $5\sqrt{2}$ .

Gram matrix

Definition Let $V$ be an inner product space over $\mathbb{F}$ with basis $\mathcal{B} = (b_1, \dots, b_n)$ . Its Gram matrix is $G \in \mathbb{F}^{n \times n}$ with entries

G_{ij} = \langle b_i, b_j \rangle.

If $x$ and $y$ have coordinate columns $\xi = [x]_{\mathcal{B}}$ and $\eta = [y]_{\mathcal{B}}$ , expanding both sums gives

\begin{aligned} \langle x, y \rangle &= \sum_{i, j} \overline{\xi_i}\,\eta_j \langle b_i, b_j \rangle \\[0.4em] &= \xi^{*}G\eta. \end{aligned}

The conjugation falls on the whole first coordinate column. The matrix $G$ is Hermitian, since $G_{ji} = \overline{G_{ij}}$ , and positive definite, since $\xi^{*}G\xi = \lVert x \rVert^2 > 0$ for $\xi \ne 0$ . Conversely, every Hermitian positive definite $G \in \mathbb{F}^{n \times n}$ defines an inner product on $\mathbb{F}^n$ by $\langle \xi, \eta \rangle_G = \xi^{*}G\eta$ . The basis is orthonormal exactly when $G = I$ , and a change of basis with matrix $S$ as above replaces $G$ by $S^{*}GS$ .

In plain words The Gram matrix records how the basis vectors pair with each other, and from it the inner product of any two vectors can be computed from their coordinates alone. A new basis can give a different Gram matrix for the same inner product.

Example In $\mathbb{R}^2$ with the dot product, take $b_1 = (1, 0)$ and $b_2 = (1, 1)$ . Then

G = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}.

The coordinate column $\xi = (0, 1)$ describes $b_2 = (1, 1)$ , and $\xi^{\mathsf{T}}G\xi = 2$ , its true squared length, not $1$ . In $\mathbb{C}^2$ with the standard inner product, the basis $b_1 = (1, i)$ , $b_2 = (1, 1)$ gives

G = \begin{bmatrix} 2 & 1 - i \\ 1 + i & 2 \end{bmatrix},

which is Hermitian. The coordinates $\xi = (1, -1)$ describe $b_1 - b_2 = (0, -1 + i)$ , whose squared norm is $2$ , and indeed $\xi^{*}G\xi = 2 - (1 - i) - (1 + i) + 2 = 2$ .

Check yourself

In $\mathbb{C}^2$ with the standard inner product, let $x = (2, i)$ and $y = (1 + i, 1)$ . Find $\langle x, y \rangle$ , $\langle y, x \rangle$ , $\lVert x \rVert$ , $\lVert y \rVert$ and $\langle ix, y \rangle$ .

Show the answer

Conjugate the first vector, then multiply and add,

\begin{aligned} \langle x, y \rangle &= 2(1 + i) + (-i)(1) \\[0.4em] &= 2 + i. \end{aligned}

By conjugate symmetry $\langle y, x \rangle = 2 - i$ . The norms are $\lVert x \rVert = \sqrt{4 + 1} = \sqrt{5}$ and $\lVert y \rVert = \sqrt{2 + 1} = \sqrt{3}$ . A factor $i$ in the first slot comes out conjugated, so $\langle ix, y \rangle = -i(2 + i) = 1 - 2i$ . Watch for the unconjugated product $x^{\mathsf{T}}y = 2 + 3i$ , which is not the inner product.

In $\mathbb{C}^2$ let $x = (1, 0)$ and $y = (i, 0)$ . Check that $\lVert x + y \rVert^2 = \lVert x \rVert^2 + \lVert y \rVert^2$ . Does it follow that $x$ and $y$ are orthogonal?

Show the answer

The sum is $(1 + i, 0)$ , with squared norm $\lvert 1 + i \rvert^2 = 2 = 1 + 1$ . Yet $\langle x, y \rangle = \overline{1} \cdot i = i \ne 0$ , so the vectors are not orthogonal. In fact $y = ix$ lies on the same complex line as $x$ . The expansion

\begin{aligned} \lVert x + y \rVert^2 &= \lVert x \rVert^2 + \lVert y \rVert^2 \\[0.4em] &\quad + 2\operatorname{Re}\langle x, y \rangle \end{aligned}

shows that the squared norms add exactly when $\operatorname{Re}\langle x, y \rangle = 0$ . Over $\mathbb{C}$ that is weaker than orthogonality, so the converse of the Pythagorean identity fails.

Which of these define an inner product on $\mathbb{C}^2$ ? (a) $\langle x, y \rangle = x^{\mathsf{T}}y$ . (b) $\langle x, y \rangle = x^{*}Hy$ with $H$ having rows $(2, i)$ and $(-i, 1)$ . (c) $\langle x, y \rangle = x^{*}Dy$ with $D = \operatorname{diag}(1, 0)$ .

Show the answer

Rule (a) is not an inner product. It is bilinear, and the nonzero vector $(i, 0)$ gives $i^2 = -1$ .

Rule (b) is an inner product. The slot rules hold for $x^{*}My$ with any $2 \times 2$ matrix $M$ , by matrix multiplication. This $H$ is also Hermitian, because conjugating and transposing returns rows $(2, i)$ and $(-i, 1)$ , so conjugate symmetry holds too. For positivity, expand $\lvert z_2 - iz_1 \rvert^2$ to get

z^{*}Hz = \lvert z_1 \rvert^2 + \lvert z_2 - iz_1 \rvert^2.

Both terms are nonnegative. If the sum is $0$ , then $z_1 = 0$ and then $z_2 = 0$ , so the form is positive definite.

Rule (c) is not an inner product. The matrix is Hermitian, but the nonzero vector $(0, 1)$ has squared length $0$ , so positive definiteness fails.

Complex Inner Products

Key terms

Conjugate transpose

Complex inner product

Sesquilinear form

Positive definiteness

Norm from an inner product

Gram matrix

Check yourself

How this lesson connects

Builds on

Used later in