Probability and Odds

Definition For an event $A$ with probability $p = P(A)$, the odds in favor of $A$ are the ratio $P(A) : P(A^{c})$, that is $p : (1 - p)$. When $p < 1$, the odds have the numerical value

which can be any number in $[0, \infty)$. A ratio $a : b$ with $b > 0$ has numerical value $a/b$, and multiplying both parts by the same positive number gives the same odds. Because $p/(1 - p)$ increases strictly with $p$ on $[0, 1)$, a higher probability there means higher odds. At $p = 1$ the ratio is $1 : 0$, which has no finite numerical value. Calling it infinite odds is a convention, not a real-number quotient.

Definition The odds against an event $A$ reverse the comparison, as the ratio $P(A^{c}) : P(A)$. When $p = P(A) > 0$, they have the numerical value $(1 - p)/p$. If $0 < p < 1$, the odds against are the reciprocal of the odds in favor. At $p = 0$ the odds in favor are $0 : 1$, with value $0$, while the odds against are $1 : 0$, with no finite value. For ordinary finite calculations in both directions, keep $p$ strictly between $0$ and $1$.

Definition If the odds in favor of $A$ are $a : b$ with $a \ge 0$, $b \ge 0$ and $a + b > 0$, then

since the event has $a$ parts out of $a + b$ equal parts. If the odds in favor have numerical value $o \ge 0$, then

This inverts $o = p/(1 - p)$, because solving $o(1 - p) = p$ for $p$ gives $p = o/(1 + o)$. It maps $[0, \infty)$ onto $[0, 1)$. Under the same conditions on $a$ and $b$, odds of $a : b$ against give $P(A) = b/(a + b)$.

Definition Let $H$ be a hypothesis with $0 < P(H) < 1$ and $E$ an event with $P(E \mid H^{c}) > 0$. The likelihood ratio of $E$ for $H$ against $H^{c}$ is

A ratio above $1$ means that $E$ is more probable under $H$, and observing $E$ raises the odds of $H$. A ratio below $1$ lowers them, and a ratio of exactly $1$ leaves them unchanged. Each observation has its own ratio. For the opposite result $E^{c}$, assuming $P(E \mid H^{c}) < 1$,

which in general is neither $1/\operatorname{LR}(E)$ nor $1 - \operatorname{LR}(E)$. If $P(E \mid H^{c}) = 0$ while $P(E \mid H) > 0$, the finite ratio does not exist, and observing $E$ rules out $H^{c}$.

Definition For a hypothesis $H$ with $0 < P(H) < 1$, the prior odds of $H$ and, after evidence $E$, the posterior odds of $H$ are

If $P(E \mid H^{c}) > 0$, then $P(E) > 0$ and $P(H^{c} \mid E) > 0$, and the odds form of Bayes’ theorem states

To see it, write Bayes’ theorem for $H$ and for $H^{c}$. Both fractions have the denominator $P(E)$, which cancels in their ratio and leaves the ratio of likelihoods times the ratio of priors. For a second observation $E_2$ after $E_1$, the same rule applies to the current odds $O(H \mid E_1)$ with the conditional likelihood ratio

assuming the conditioning events have positive probability and the denominator is positive. Then $O(H \mid E_1 \cap E_2) = \operatorname{LR}_2\, O(H \mid E_1)$. The ratio $\operatorname{LR}_2$ equals the single-observation ratio $P(E_2 \mid H)/P(E_2 \mid H^{c})$ when the two observations are conditionally independent given $H$ and given $H^{c}$.