Multivariate distributions and dependence

Definition For random variables $X$ and $Y$ on one probability space $(\Omega, \mathcal{F}, \mathbb{P})$, the joint distribution is the law of the random vector $(X, Y)$, the probability measure that gives each Borel set $B \subseteq \mathbb{R}^2$ the probability $\mathbb{P}((X, Y) \in B)$. It is determined by the joint CDF

The marginal CDFs follow from it, since $F_X(x) = \lim_{y \to \infty} F_{X,Y}(x, y)$ and similarly for $Y$, but the marginals do not determine the joint law. The variables are independent exactly when $F_{X,Y}(x, y) = F_X(x)\,F_Y(y)$ for all $x$ and $y$, or equivalently when $\mathbb{P}(X \in A, Y \in B) = \mathbb{P}(X \in A)\,\mathbb{P}(Y \in B)$ for all Borel sets $A$ and $B$. A random vector with $n$ coordinates is handled the same way with $n$ thresholds.

Definition For square-integrable $X$ and $Y$ with means $\mu_X$ and $\mu_Y$, the covariance is

If both standard deviations $\sigma_X = \sqrt{\operatorname{Var}(X)}$ and $\sigma_Y$ are positive, the correlation is

The Cauchy-Schwarz inequality applied to the centered variables gives $\lvert \operatorname{Cov}(X, Y) \rvert \le \sigma_X \sigma_Y$, so $-1 \le \rho_{X,Y} \le 1$. Equality $\lvert \rho_{X,Y} \rvert = 1$ holds exactly when $Y - \mu_Y = c\,(X - \mu_X)$ almost surely for a constant $c \ne 0$, and the sign of $c$ is the sign of $\rho_{X,Y}$. If one variance is zero, that variable is almost surely constant, its covariance with any variable is $0$, and the correlation is undefined rather than zero. Covariance is bilinear with $\operatorname{Cov}(X, X) = \operatorname{Var}(X)$, and correlation does not change when $X$ is replaced by $a + bX$ with $b > 0$. For a random vector $X \in \mathbb{R}^n$ with finite second moments, the covariance matrix $\Sigma$ with entries $\Sigma_{ij} = \operatorname{Cov}(X_i, X_j)$ is symmetric and positive semidefinite, because $w^{\top} \Sigma w = \operatorname{Var}(w^{\top} X) \ge 0$ for every $w \in \mathbb{R}^n$.

Definition If $X$ and $Y$ are independent and square-integrable, then $\mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y]$, so $\operatorname{Cov}(X, Y) = 0$. Variables with zero covariance are called uncorrelated. The converse fails. Zero covariance is the single equation $\mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y]$, while independence requires $\mathbb{P}(X \in A, Y \in B) = \mathbb{P}(X \in A)\,\mathbb{P}(Y \in B)$ for all Borel sets $A$ and $B$. Taking indicator functions shows that independence is the same as $\operatorname{Cov}(g(X), h(Y)) = 0$ for all bounded Borel functions $g$ and $h$. To prove dependence, one pair of events whose joint probability differs from the product of their probabilities is enough.

Definition A random vector $X \in \mathbb{R}^n$ is jointly Gaussian if $a^{\top} X$ is normal for every $a \in \mathbb{R}^n$, where a constant counts as a normal variable with variance zero. Then $X$ has a mean vector $\mu$ and a symmetric positive semidefinite covariance matrix $\Sigma$, written $X \sim N(\mu, \Sigma)$, and for every $a \in \mathbb{R}^n$

The law is determined by $\mu$ and $\Sigma$ through the characteristic function, for $u \in \mathbb{R}^n$,

Conversely, for any $\mu$ and any positive semidefinite $\Sigma$, if $Z$ has independent standard normal coordinates and $L$ is a matrix with $L L^{\top} = \Sigma$, then $\mu + LZ \sim N(\mu, \Sigma)$. When $\Sigma$ is positive definite, the Cholesky factor is one such $L$. If $\Sigma$ is singular, $X$ lies almost surely in the affine subspace $\mu + \operatorname{Col}\Sigma$, which has lower dimension, and $X$ has no density on $\mathbb{R}^n$. If $X = (X_1, X_2)$ is jointly Gaussian and the cross-covariance block $\operatorname{Cov}(X_1, X_2)$ is zero, then $X_1$ and $X_2$ are independent. The quadratic form $u^{\top}\Sigma u$ then splits into one term for each block, so the characteristic function factors, and a characteristic function determines the law, a result stated here. Normal marginals alone do not make a vector jointly Gaussian.

Definition A bivariate copula is the joint CDF $C$ of a random vector $(U, V)$ whose coordinates are each uniform on $[0, 1]$, so $C(u, 1) = u$ and $C(1, v) = v$. If $F_X$ is continuous, then $F_X(X)$ is uniform on $[0, 1]$, which is the probability integral transform. Sklar’s theorem, stated here without proof, says that every joint CDF $F_{X,Y}$ with marginals $F_X$ and $F_Y$ can be written as

for all $x$ and $y$, with a copula $C$ that is unique on the product of the ranges of $F_X$ and $F_Y$. If both marginals are continuous, $C$ is unique on all of $[0, 1]^2$ and is the joint CDF of $(F_X(X), F_Y(Y))$. Conversely, for any copula $C$ and any univariate CDFs $F$ and $G$, the function $C(F(x), G(y))$ is a joint CDF with marginals $F$ and $G$. With continuous marginals, applying strictly increasing functions to $X$ and $Y$ leaves the copula unchanged. Examples are the independence copula $uv$, the comonotone copula $\min(u, v)$, and the Gaussian copula $C_\rho(u, v) = \Phi_\rho(\Phi^{-1}(u), \Phi^{-1}(v))$, where $\Phi_\rho$ is the bivariate standard normal CDF with correlation $\rho$. For continuous marginals, write $U = F_X(X)$ and $V = F_Y(Y)$. The upper tail dependence coefficient is

when the limit exists. If a marginal has an atom, $F_X(X)$ is not uniform and the copula is pinned down only on the ranges.

Definition Let $R = (R_1, \dots, R_n)$ be the one-period simple returns of $n$ assets, with finite second moments and covariance matrix $\Sigma$, and let $w \in \mathbb{R}^n$ be fixed weights, fractions of initial wealth with $\sum_i w_i = 1$, where a negative weight is a short position. The portfolio return $w^{\top} R$ has

For two assets with standard deviations $\sigma_1$ and $\sigma_2$ and correlation $\rho$,

The formula holds for every joint law with finite second moments, Gaussian or not. When $R \sim N(\mu, \Sigma)$, the portfolio return is $N(w^{\top}\mu, w^{\top}\Sigma w)$, so its mean and variance fix its whole distribution. Otherwise they do not determine its tail probabilities. With nonnegative weights the portfolio standard deviation is at most $\sum_i w_i \sigma_i$. For two assets with positive weights and positive standard deviations, equality holds only when $\rho = 1$.