Covariance explorer

Any two variables with finite variances have a $2 \times 2$ covariance matrix, and that matrix has a shape you can see. Set the standard deviations $\sigma_1$ and $\sigma_2$ and the correlation $\rho$, and the cloud of points, the ellipses and the principal axes move with them.

The points are one fixed random sample. Every setting reuses the same draws, so you can follow a single point as the cloud stretches and turns.

Things to try

• Set $\sigma_1 = \sigma_2 = 1$ and $\rho = 0$, or press Circle. Both ellipses become circles, $\lambda_1 = \lambda_2 = 1$, and every direction through the center is a principal axis.
• Keep $\rho = 0$, set $\sigma_1$ below $\sigma_2$ and slide it up past $\sigma_2$. The first principal axis switches from vertical to horizontal. With no correlation the principal axes are the coordinate axes, and the first one follows the larger variance.
• Push $\rho$ toward $0.95$. The total variance $\sigma_1^2 + \sigma_2^2$ does not move, but $\det(\boldsymbol\Sigma) = \sigma_1^2 \sigma_2^2 (1 - \rho^2)$ shrinks toward $0$ and the cloud flattens toward a line.
• Make $\sigma_1$ and $\sigma_2$ equal and try any nonzero $\rho$. The first axis sits at exactly $45^\circ$ when $\rho$ is positive and $-45^\circ$ when it is negative, and the eigenvalues are $\sigma_1^2 (1 + \lvert \rho \rvert)$ and $\sigma_1^2 (1 - \lvert \rho \rvert)$.
• Flip the sign of $\rho$. The ellipses and the principal axes reflect across the horizontal axis, while the eigenvalues, the trace and the determinant stay exactly the same.
• Pick a point just inside the inner ellipse and move any slider. It never crosses the ellipse, because each point keeps the same Mahalanobis distance however $\boldsymbol\Sigma$ changes. The next section shows why.

What you are seeing

With these parameters the covariance matrix is

It is positive definite because $\lvert \rho \rvert < 1$. Its eigenvalues $\lambda_1 \ge \lambda_2 > 0$ come with orthonormal eigenvectors $\mathbf e_1$ and $\mathbf e_2$, which give the spectral decomposition

For a $2 \times 2$ matrix the eigenvalues have a closed form.

Their sum is the total variance and their product is the generalized variance.

The share $\lambda_1 / (\lambda_1 + \lambda_2)$ is the proportion of the total variance along the first principal component.

The ellipses are the points at Mahalanobis distance $c$ from the mean $\mathbf 0$, with $c = 1$ for the inner one and $c = 2$ for the outer one.

Each ellipse has its axes along $\mathbf e_1$ and $\mathbf e_2$, with half lengths $c \sqrt{\lambda_1}$ and $c \sqrt{\lambda_2}$. The two colored segments are these axes for $c = 2$.

Each sample point starts as a pair $\mathbf z$ of independent standard normal values, drawn once. The plot shows $\mathbf x = \mathbf L \mathbf z$, where $\mathbf L$ is the Cholesky factor of $\boldsymbol\Sigma$.

That makes $\mathbf x$ a draw from $N_2(\mathbf 0, \boldsymbol\Sigma)$, and for every setting its squared distance is

Under this model $\mathbf x' \boldsymbol\Sigma^{-1} \mathbf x$ has a $\chi^2_2$ distribution, so

That is about $0.39$ inside the inner ellipse and $0.86$ inside the outer one. The 240 points here are one random draw, so their shares come out near those values but not exactly on them.

The multivariate statistics reference has the notation and the main results in one place, and the linear algebra refresher goes over eigenvalues and quadratic forms. The videos are on the Multivariate Statistical Analysis course page.