Probability spaces and conditional information

Definition Let $\Omega$ be a nonempty set of outcomes, also called states. A collection $\mathcal{F}$ of subsets of $\Omega$ is a sigma algebra if

Its members are the measurable events, the events to which a probability can be assigned. A sigma algebra also contains $\varnothing$ and is closed under countable intersections and under differences. The power set of $\Omega$, the collection of all subsets, is the largest sigma algebra on $\Omega$, and $\{\varnothing, \Omega\}$ is the smallest, the trivial sigma algebra. For any collection $\mathcal{C}$ of subsets, the sigma algebra $\sigma(\mathcal{C})$ generated by $\mathcal{C}$ is the smallest sigma algebra that contains $\mathcal{C}$, the intersection of all sigma algebras that contain it. When $\Omega$ is finite, closure under complements and finite unions is already enough.

Definition A probability space is a triple $(\Omega, \mathcal{F}, \mathbb{P})$ made of a nonempty outcome set $\Omega$, a sigma algebra $\mathcal{F}$ on $\Omega$ and a probability measure $\mathbb{P}$ on $\mathcal{F}$. A probability measure is a function from $\mathcal{F}$ to $[0, 1]$ with $\mathbb{P}(\Omega) = 1$ that is countably additive,

for pairwise disjoint events $A_1, A_2, \ldots \in \mathcal{F}$. It follows that $\mathbb{P}(\varnothing) = 0$, that $\mathbb{P}(A^{c}) = 1 - \mathbb{P}(A)$, that $\mathbb{P}(A) \le \mathbb{P}(B)$ when $A \subseteq B$, and that $\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B)$. On a finite $\Omega$ whose sigma algebra is the power set, a probability measure is the same thing as a list of state masses $p(\omega) \ge 0$ that add to $1$, with $\mathbb{P}(A) = \sum_{\omega \in A} p(\omega)$. An event of probability zero is a null event, and a statement holds almost surely if it fails at most on a null event. A null event need not be empty. A random variable with a continuous distribution takes each single value with probability zero, yet it always takes some value. The same pair $(\Omega, \mathcal{F})$ can carry several measures. In this course $\mathbb{P}$ is the physical measure, and later lessons add a pricing measure $\mathbb{Q}$ that in general gives the same events different probabilities.

Definition A partition of a finite set $\Omega$ is a collection of nonempty, pairwise disjoint subsets $B_1, \ldots, B_k$, its blocks, whose union is $\Omega$. The sigma algebra $\sigma(B_1, \ldots, B_k)$ that it generates consists of all unions of blocks, the empty union $\varnothing$ included, so it has $2^k$ events. Conversely, every sigma algebra $\mathcal{G}$ on a finite $\Omega$ is generated by exactly one partition, the partition into its atoms, where the atom containing a state $\omega$ is the intersection of all events of $\mathcal{G}$ that contain $\omega$. An information partition describes an observer who learns which block contains the true state and nothing more, so that the observer’s information is the sigma algebra $\mathcal{G}$ generated by the partition. An event $A$ is $\mathcal{G}$-measurable, meaning that the observer can always tell whether $A$ occurred, exactly when it is a union of blocks, that is, when each block lies inside $A$ or is disjoint from it. A partition $\pi'$ refines a partition $\pi$ when every block of $\pi'$ lies inside a block of $\pi$, which holds exactly when $\sigma(\pi) \subseteq \sigma(\pi')$. A filtration is an increasing family $\mathcal{F}_0 \subseteq \mathcal{F}_1 \subseteq \cdots \subseteq \mathcal{F}$ of sigma algebras indexed by dates, and $\mathcal{F}_t$ is the information available at date $t$. On a finite $\Omega$ a filtration amounts to a sequence of partitions, each refining the one before.

Definition Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space and $B \in \mathcal{F}$ an event with $\mathbb{P}(B) > 0$. The conditional probability of an event $A \in \mathcal{F}$ given $B$ is

The map $A \mapsto \mathbb{P}(A \mid B)$ is again a probability measure on $(\Omega, \mathcal{F})$, and it gives $B$ probability $1$, so the conditional probabilities of the blocks of any partition add to $1$. Rearranging gives the multiplication rule $\mathbb{P}(A \cap B) = \mathbb{P}(A \mid B)\,\mathbb{P}(B)$. Events $A$ and $B$ are independent if $\mathbb{P}(A \cap B) = \mathbb{P}(A)\,\mathbb{P}(B)$, which for $\mathbb{P}(B) > 0$ is equivalent to $\mathbb{P}(A \mid B) = \mathbb{P}(A)$. Disjoint events with positive probabilities are never independent, since their intersection has probability $0$ while the product of their probabilities is positive. Given a finite partition $B_1, \ldots, B_k$ whose blocks all have positive probability, the conditional probability of $A$ given this information is the random variable equal to $\mathbb{P}(A \mid B_j)$ on each block $B_j$. It is measurable with respect to the information, and averaging it with the block probabilities gives back $\mathbb{P}(A)$, which is the law of total probability. If $\mathbb{P}(B) = 0$ the ratio is undefined. Conditioning on a null event, such as a continuous variable taking one exact value, needs densities or regular conditional distributions with their own assumptions and lies outside this lesson.

Definition Let $A$ and $B$ be events with $\mathbb{P}(A) > 0$, $\mathbb{P}(A^{c}) > 0$ and $\mathbb{P}(B) > 0$. Then

where, by the law of total probability,

The first formula holds because the joint mass can be written in two ways, $\mathbb{P}(A \cap B) = \mathbb{P}(B \mid A)\,\mathbb{P}(A) = \mathbb{P}(A \mid B)\,\mathbb{P}(B)$, and the second because $B$ is the disjoint union of $B \cap A$ and $B \cap A^{c}$. More generally, if $A_1, \ldots, A_k$ partition $\Omega$ into events of positive probability and $\mathbb{P}(B) > 0$, then for each $j$

The prior $\mathbb{P}(A)$, the likelihoods $\mathbb{P}(B \mid A)$ and $\mathbb{P}(B \mid A^{c})$ and the posterior $\mathbb{P}(A \mid B)$ are all probabilities under one measure. The theorem is an identity within a specified model. It does not check that the prior and the error rates were estimated well, and it does not decide which measure belongs in a valuation, where a pricing measure $\mathbb{Q}$ may give default a different probability from the one under $\mathbb{P}$.

Definition The base rate of an event $A$ is its unconditional probability $\mathbb{P}(A)$ in the model before the signal is observed, the prior in Bayes’ theorem. In a population model it is the fraction of members with the property $A$. When $0 < \mathbb{P}(A) < 1$ and $\mathbb{P}(B \mid A^{c}) > 0$, Bayes’ theorem in odds form reads

so the posterior odds equal the likelihood ratio times the prior odds. The likelihood ratio $\mathbb{P}(B \mid A)/\mathbb{P}(B \mid A^{c})$ describes the signal alone, and the base rate enters only through the prior odds. For a fixed positive likelihood ratio the posterior increases strictly with the base rate and tends to $0$ as the base rate tends to $0$. Base-rate neglect is the error of judging the posterior from the signal alone, for example by reading $\mathbb{P}(B \mid A)$ as if it were $\mathbb{P}(A \mid B)$, or the likelihood ratio as if it were the posterior odds. At the other extreme, a signal without false positives, one with $\mathbb{P}(B \mid A^{c}) = 0$ and $\mathbb{P}(B \mid A) > 0$, gives $\mathbb{P}(A \mid B) = 1$ for every base rate $\mathbb{P}(A) > 0$.