Poisson and Normal Approximations

Definition The rare-event regime for a count $X \sim \operatorname{Bin}(n, p)$ is the setting of many trials, each with a small success probability, and a moderate mean $np$. The limit theorem behind it reads as follows. Let $\lambda > 0$, and for each $n \ge 1$ let $X_n \sim \operatorname{Bin}(n, p_n)$ with $np_n \to \lambda$ as $n \to \infty$. Then for every fixed $k = 0, 1, 2, \dots$

In this regime $\operatorname{Bin}(n, p)$ is approximated by $\operatorname{Pois}(np)$, the Poisson law with the same mean. Le Cam’s inequality bounds the error at a fixed $n$. If $Y \sim \operatorname{Pois}(np)$, then $\lvert P(X \in A) - P(Y \in A) \rvert \le np^{2}$ for every set $A$ of counts.

Definition Fix $p$ with $0 < p < 1$ and let $X_n \sim \operatorname{Bin}(n, p)$. The count $X_n$ is a sum of $n$ independent Bernoulli indicators with finite positive variance $p(1 - p)$, so the central limit theorem gives, for every real $z$,

The normal approximation replaces probabilities of $X \sim \operatorname{Bin}(n, p)$ by those of $Y \sim N\bigl(np, np(1 - p)\bigr)$, the normal law with the same mean and variance. The theorem is a limit statement and gives no error bound at a fixed $n$. Screening rules, such as asking for both $np$ and $n(1 - p)$ to be at least $10$, are rough guides rather than theorems, and no single threshold makes every event accurate.

Definition Let $X$ take integer values and let $Y$ be a continuous variable used to approximate it, such as a normal variable with the same mean and variance. First write the event as a set of consecutive integers. Then let each integer $k$ stand for the interval from $k - \tfrac12$ to $k + \tfrac12$. For integers $a \le b$ this gives

In the same way $P(X \ge a)$ is approximated by $P(Y > a - \tfrac12)$ and $P(X \le b)$ by $P(Y < b + \tfrac12)$. A strict inequality is rewritten first, so $X > 13$ means $X \ge 14$ and becomes $Y > 13.5$, while $X \ge 13$ becomes $Y > 12.5$.

Definition Let $p_*$ be the exact probability of an event under the model and $a$ an approximation to it. The symbol $p_*$ is the target probability, not the success probability $p$ of a single trial. The absolute error is $\lvert a - p_* \rvert$, measured on the probability scale. When $p_* > 0$ the relative error is

often written as a percent. When $p_* = 0$ the relative error is not defined.

Definition Tail accuracy is the accuracy of an approximation for a tail probability such as $P(X \ge t)$ or $P(X \le t)$ with $t$ far from the mean. Tail probabilities are small, so tail accuracy is judged by relative error. An error found for one event need not carry over to another event, even with the same parameters, so each approximation is checked for the event actually asked about.