Quadratic Forms and Positive Definiteness

Definition A quadratic form on $\mathbb{R}^n$ is a function $q$ from $\mathbb{R}^n$ to $\mathbb{R}$ of the form

for some real $n \times n$ matrix $A$. The product $A\mathbf{x}$ is a vector, and its dot product with $\mathbf{x}$ is a single number, so $q$ assigns a scalar to each vector, unlike the linear map $\mathbf{x} \mapsto A\mathbf{x}$. Every term has degree exactly $2$ in the entries of $\mathbf{x}$, so $q(c\mathbf{x}) = c^2 q(\mathbf{x})$ for every real $c$. In particular $q(\mathbf{0}) = 0$ and $q(-\mathbf{x}) = q(\mathbf{x})$. A quadratic form is generally not linear.

Definition Every quadratic form can be written as $q(\mathbf{x}) = \mathbf{x}^{\top}A\mathbf{x}$ with $A$ symmetric, and this symmetric matrix is unique. It is called the matrix of the quadratic form. If $q(\mathbf{x}) = \mathbf{x}^{\top}M\mathbf{x}$ for some real square matrix $M$, split $M = A + K$ into its symmetric part $A = \frac{1}{2}(M + M^{\top})$ and its skew-symmetric part $K = \frac{1}{2}(M - M^{\top})$. The skew part contributes nothing, because the number $\mathbf{x}^{\top}K\mathbf{x}$ equals its own transpose $\mathbf{x}^{\top}K^{\top}\mathbf{x} = -\mathbf{x}^{\top}K\mathbf{x}$ and is therefore $0$. The symmetric matrix is unique because its entries are determined by values of $q$, namely $a_{ii} = q(\mathbf{e}_i)$ and, for $i \ne j$,

A cross term of $q$ is a term $k\,x_i x_j$ with $i \ne j$. In the symmetric matrix its coefficient is split equally between two positions, $a_{ij} = a_{ji} = k/2$, while the coefficient of $x_i^2$ is $a_{ii}$. In two variables,

Definition Let $q(\mathbf{x}) = \mathbf{x}^{\top}A\mathbf{x}$ with $A$ real symmetric, and let $A = QDQ^{\top}$ be an orthogonal diagonalization with orthonormal eigenvectors $\mathbf{q}_1, \dots, \mathbf{q}_n$ and eigenvalues $\lambda_1, \dots, \lambda_n$. The lines through the origin spanned by $\mathbf{q}_1, \dots, \mathbf{q}_n$ are the principal axes of $q$. When an eigenvalue is repeated, the axes inside its eigenspace can be chosen in many ways, and any orthonormal choice works. In the coordinates $\mathbf{z} = Q^{\top}\mathbf{x}$ along these axes, so that $\mathbf{x} = Q\mathbf{z}$ and $z_i = \mathbf{q}_i \cdot \mathbf{x}$, the form has no cross terms,

Because $Q$ is orthogonal, the new coordinates measure lengths and angles exactly as the old ones do, so a level set keeps its shape and size and is only described differently. When $n = 2$ and $\lambda_1, \lambda_2 > 0$, the level curve $q = 1$ is an ellipse with semi-axes of length $1/\sqrt{\lambda_1}$ along $\mathbf{q}_1$ and $1/\sqrt{\lambda_2}$ along $\mathbf{q}_2$.

Definition A real symmetric $n \times n$ matrix $A$, and its quadratic form $q(\mathbf{x}) = \mathbf{x}^{\top}A\mathbf{x}$, are positive definite if $q(\mathbf{x}) > 0$ for every nonzero $\mathbf{x}$ in $\mathbb{R}^n$, and negative definite if $q(\mathbf{x}) < 0$ for every nonzero $\mathbf{x}$. The zero vector is excluded because $q(\mathbf{0}) = 0$ always. The eigenvalue test says that $A$ is positive definite if and only if all its eigenvalues are positive, and negative definite if and only if all its eigenvalues are negative. To prove the first statement, write $q = \lambda_1 z_1^2 + \cdots + \lambda_n z_n^2$ with $\mathbf{z} = Q^{\top}\mathbf{x}$. Since $Q$ is invertible, $\mathbf{x} \ne \mathbf{0}$ exactly when $\mathbf{z} \ne \mathbf{0}$. If every $\lambda_i > 0$, all terms are nonnegative and at least one is positive, so $q(\mathbf{x}) > 0$. Conversely, a unit eigenvector $\mathbf{q}_i$ gives

so a positive definite form has every $\lambda_i > 0$. The negative case follows by applying this to $-A$.

Definition A real symmetric $n \times n$ matrix $A$, and its form $q$, are positive semidefinite if $q(\mathbf{x}) \ge 0$ for every $\mathbf{x}$ in $\mathbb{R}^n$, and negative semidefinite if $q(\mathbf{x}) \le 0$ for every $\mathbf{x}$. The same argument in eigen-coordinates shows that $A$ is positive semidefinite if and only if all its eigenvalues are nonnegative, and negative semidefinite if and only if all are nonpositive. The classes are inclusive. A positive definite matrix is also positive semidefinite, and the zero matrix is both positive and negative semidefinite but neither positive nor negative definite. For a positive semidefinite $A$, the value $q(\mathbf{x})$ is $0$ exactly when $A\mathbf{x} = \mathbf{0}$. Indeed $\lambda_1 z_1^2 + \cdots + \lambda_n z_n^2 = 0$ with every $\lambda_i \ge 0$ forces $z_i = 0$ whenever $\lambda_i > 0$, so $\mathbf{x}$ is a combination of eigenvectors for the eigenvalue $0$, and these lie in $\operatorname{Null} A$. As a consequence, a positive semidefinite matrix is positive definite exactly when it is invertible.

Definition A real symmetric matrix $A$, and its form $q$, are indefinite if $q$ takes both signs, that is, $q(\mathbf{u}) > 0$ and $q(\mathbf{w}) < 0$ for some vectors $\mathbf{u}$ and $\mathbf{w}$. This happens if and only if $A$ has at least one positive and at least one negative eigenvalue. The signs of the eigenvalues therefore classify every real symmetric matrix.

The zero matrix belongs to both semidefinite rows. Two vectors with values of opposite signs prove that a form is indefinite without computing any eigenvalue. For an indefinite form, $q(\mathbf{x}) = 0$ does not mean $A\mathbf{x} = \mathbf{0}$, since positive and negative terms can cancel.