Rayleigh Quotients

Definition Let $\mathbb{F}$ be $\mathbb{R}$ or $\mathbb{C}$, and let $A \in \mathbb{F}^{n \times n}$ be Hermitian (real symmetric when $\mathbb{F} = \mathbb{R}$). The Rayleigh quotient of $A$ is the function defined for nonzero $x \in \mathbb{F}^n$ by

The denominator is $\lVert x \rVert^2 > 0$, and the numerator is real because $A$ is Hermitian, so $R_A(x)$ is a real number. If $Ax = \lambda x$ with $x \ne 0$, then $R_A(x) = \lambda\, x^{*}x / x^{*}x = \lambda$, so every eigenvalue of $A$ is a value of the quotient.

Definition For every nonzero $x \in \mathbb{F}^n$ and every nonzero scalar $c \in \mathbb{F}$,

So the quotient is constant along each line through the origin, with the origin itself removed. In particular $R_A(x) = R_A(x / \lVert x \rVert)$, and $R_A(u) = u^{*}Au$ when $\lVert u \rVert = 1$, so the values of the quotient are exactly the values of $u^{*}Au$ on the unit sphere. Over $\mathbb{C}$ the scalar $c$ may be complex, so all the phase multiples $e^{i\theta}x$ give the same value.

Definition Let $A \in \mathbb{F}^{n \times n}$ be Hermitian, and let $q_1, \dots, q_n$ be an orthonormal basis of $\mathbb{F}^n$ with $Aq_j = \lambda_j q_j$ and every $\lambda_j$ real, as the spectral theorem provides. Write a nonzero $x$ as $x = c_1q_1 + \cdots + c_nq_n$ with $c_j = q_j^{*}x$. Then $Ax = \sum_j \lambda_j c_j q_j$, orthonormality removes every mixed term, and

Hence

The weights are nonnegative and add up to $1$, so the quotient is a weighted average of the eigenvalues. The eigenvalues are the same for every input, and only the weights depend on $x$. For a unit vector the weights are simply $\lvert c_j \rvert^2$. The argument works over $\mathbb{C}$ because each $\lvert c_j \rvert^2$ is a nonnegative real number.

Definition Let $A \in \mathbb{F}^{n \times n}$ be Hermitian with eigenvalues ordered as $\lambda_1 \le \lambda_2 \le \cdots \le \lambda_n$, repeated according to multiplicity. Then

for every nonzero $x \in \mathbb{F}^n$, and both bounds are attained, so

With the weights $w_j$ from the eigenvector expansion, the lower bound follows from $R_A(x) - \lambda_1 = \sum_j w_j(\lambda_j - \lambda_1)$, a sum of nonnegative terms, and the upper bound from $\lambda_n - R_A(x) = \sum_j w_j(\lambda_n - \lambda_j)$ in the same way. A unit eigenvector for $\lambda_1$ gives the value $\lambda_1$, and one for $\lambda_n$ gives $\lambda_n$, so the bounds are sharp. The Hermitian hypothesis is needed. The real form $x^{\mathsf{T}}Bx$ of a real matrix $B$ only detects its symmetric part, so the eigenvalues of a nonsymmetric $B$ need not bound it. A complex matrix that is not Hermitian can give nonreal values, as $iI$ does, with quotient $i$ at every nonzero vector, and then a minimum does not even make sense.

Definition Let $A \in \mathbb{F}^{n \times n}$ be Hermitian with eigenvalues $\lambda_1 \le \cdots \le \lambda_n$. For nonzero $x$,

and likewise $R_A(x) = \lambda_n$ exactly when $Ax = \lambda_n x$. In the sum $R_A(x) - \lambda_1 = \sum_j w_j(\lambda_j - \lambda_1)$, equality with $0$ forces $w_j = 0$, that is $c_j = 0$, for every $j$ with $\lambda_j > \lambda_1$, so $x$ lies in the eigenspace $E_{\lambda_1} = \ker(A - \lambda_1 I)$. Conversely every nonzero vector of that eigenspace gives $\lambda_1$. The unit minimizers are therefore exactly the unit vectors of $E_{\lambda_1}$, whose dimension is the multiplicity $m$ of $\lambda_1$. When $m = 1$ they are $\pm q_1$ over $\mathbb{R}$, or $e^{i\theta}q_1$ over $\mathbb{C}$. When $m \ge 2$ they fill the whole unit sphere of an $m$-dimensional subspace, no single eigenvector is preferred, and any orthonormal basis of the eigenspace can serve in the expansion. The same holds for the maximizers and $E_{\lambda_n}$.