Expectation conditional expectation and variance

Definition Fix a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ and write $\mathbb{E}$ for expectation under $\mathbb{P}$. On a finite sample space $\Omega = \{\omega_1, \dots, \omega_n\}$ with $p_j = \mathbb{P}(\{\omega_j\})$,

In general $\mathbb{E}[X] = \int_\Omega X\,d\mathbb{P}$ for an integrable $X$ (next card), which equals $\int x\,f_X(x)\,dx$ when $X$ has a density $f_X$. For integrable $X$ and $Y$ and real numbers $a$ and $b$,

with no assumption on the dependence between $X$ and $Y$. When $\mathbb{E}[X^2] < \infty$ the variance is

and for $X$ and $Y$ with finite second moments, with $\operatorname{Cov}(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])]$,

The cross term vanishes when the covariance is zero, for example for independent $X$ and $Y$, and adding variances without it is wrong in general. The same formulas hold under any other probability measure, such as a pricing measure $\mathbb{Q}$, with that measure’s weights, so an expectation always belongs to a measure.

Definition A random variable $X$ is integrable if $\mathbb{E}[\lvert X \rvert] < \infty$. Then $\mathbb{E}[X] = \mathbb{E}[X^{+}] - \mathbb{E}[X^{-}]$ is a finite number, where $X^{+} = \max(X, 0)$ and $X^{-} = \max(-X, 0)$. For $X \ge 0$ the expectation is always defined, possibly as $+\infty$, and it can be computed from the tail,

A random variable is square-integrable if $\mathbb{E}[X^2] < \infty$, which implies integrability because $\lvert x \rvert \le 1 + x^2$. A variance needs a finite second moment, and a covariance is defined here for two square-integrable variables. On a finite sample space every random variable is square-integrable, but a continuous model with heavy tails can fail either condition.

Definition Let the available information be a partition of $\Omega$ into events $A_1, \dots, A_k$ with $\mathbb{P}(A_i) > 0$, and let $\mathcal{G}$ be the $\sigma$-algebra of all unions of these events. For an integrable $X$, the conditional expectation $\mathbb{E}[X \mid \mathcal{G}]$ is the random variable that equals

on each $A_i$. It is $\mathcal{G}$-measurable, because it is constant on every group, and for every $A \in \mathcal{G}$

For $A = A_i$ both sides equal $\mathbb{E}[X\,\mathbf{1}_{A_i}]$, since multiplying $m_i$ by $\mathbb{P}(A_i)$ undoes the division, and a union of groups adds these equalities. In general, for a $\sigma$-algebra $\mathcal{G} \subseteq \mathcal{F}$ and an integrable $X$, there is a random variable $Y$ that is $\mathcal{G}$-measurable and integrable and satisfies $\mathbb{E}[\mathbf{1}_{A} Y] = \mathbb{E}[\mathbf{1}_{A} X]$ for every $A \in \mathcal{G}$, and any two such variables are equal almost surely. This $Y$ is $\mathbb{E}[X \mid \mathcal{G}]$. The existence of such a $Y$ follows from the Radon-Nikodym theorem, a result stated here without proof. If a group had probability zero, the division would be impossible, and any value on that group would give another version of the same conditional expectation, which is why uniqueness holds only almost surely.

Definition Let $\mathcal{H} \subseteq \mathcal{G} \subseteq \mathcal{F}$ be $\sigma$-algebras and let $X$ be integrable. Then, almost surely,

To prove it, call the left side $Y$. It is $\mathcal{H}$-measurable and integrable. Every $A \in \mathcal{H}$ also lies in $\mathcal{G}$, so applying the defining property twice gives $\mathbb{E}[\mathbf{1}_{A} Y] = \mathbb{E}[\mathbf{1}_{A}\,\mathbb{E}[X \mid \mathcal{G}]] = \mathbb{E}[\mathbf{1}_{A} X]$. Thus $Y$ satisfies the definition of $\mathbb{E}[X \mid \mathcal{H}]$, and uniqueness gives the identity. With the trivial $\sigma$-algebra $\mathcal{H} = \{\varnothing, \Omega\}$ it becomes $\mathbb{E}\bigl[\mathbb{E}[X \mid \mathcal{G}]\bigr] = \mathbb{E}[X]$. The inclusion $\mathcal{H} \subseteq \mathcal{G}$ is a real hypothesis. For two unrelated information sets the identity fails in general.

Definition Let $\mathbb{E}[X^2] < \infty$, let $\mathcal{G} \subseteq \mathcal{F}$ be a $\sigma$-algebra, and write $m = \mathbb{E}[X \mid \mathcal{G}]$ and $\operatorname{Var}(X \mid \mathcal{G}) = \mathbb{E}[(X - m)^2 \mid \mathcal{G}]$. Then

To prove it, write $X - \mathbb{E}[X] = (X - m) + (m - \mathbb{E}[X])$, square, and take expectations. The variable $m$ has a finite second moment, since $\mathbb{E}[m^2] \le \mathbb{E}[X^2]$ by the conditional form of Jensen’s inequality, stated here. Write $c = m - \mathbb{E}[X]$. The cross term vanishes,

because $c$ is $\mathcal{G}$-measurable with a finite second moment, so it can be taken out of the conditional expectation, and $\mathbb{E}[X - m \mid \mathcal{G}] = 0$. On a partition this step is direct, since $m - \mathbb{E}[X]$ is constant on each group and the deviations $X - m_i$ average to zero within the group. The tower property turns $\mathbb{E}[(X - m)^2]$ into $\mathbb{E}[\operatorname{Var}(X \mid \mathcal{G})]$, and $\mathbb{E}[m] = \mathbb{E}[X]$ makes $\mathbb{E}[(m - \mathbb{E}[X])^2]$ equal to $\operatorname{Var}(m)$.

Definition For an information partition $A_1, \dots, A_k$ with $\mathbb{P}(A_i) > 0$, group means $m_i$ and group variances

the within-group variance is $\sum_i \mathbb{P}(A_i)\,v_i = \mathbb{E}[\operatorname{Var}(X \mid \mathcal{G})]$ and the between-group variance is $\sum_i \mathbb{P}(A_i)\,(m_i - \mathbb{E}[X])^2 = \operatorname{Var}(\mathbb{E}[X \mid \mathcal{G}])$. Both are nonnegative and they add up to $\operatorname{Var}(X)$. The within-group variance is the smallest mean squared error of any forecast that uses the information. For square-integrable $X$ and every square-integrable $\mathcal{G}$-measurable $Z$, the cross term argument of the previous card gives

with $m = \mathbb{E}[X \mid \mathcal{G}]$, so $Z = m$ is the best forecast in this sense. Since a larger $\sigma$-algebra allows more forecasts, more information can only lower the within-group variance or leave it unchanged, and the between-group variance moves the opposite way. With no information the within-group variance is $\operatorname{Var}(X)$, and when every state of a finite space is its own group of positive probability it is $0$.