Bernoulli, Binomial and Hypergeometric Models, statistics lesson

An inspector examines four items and counts how many carry a flag. In one setting a process flags each item independently with probability $0.2$ . In the other, the four items are drawn without replacement from a batch of ten that contains exactly two flagged items. Both start from the same 20 percent, yet the chance of exactly two flagged items is $0.1536$ in the first setting and $\tfrac{2}{15} \approx 0.1333$ in the second. This lesson builds the Bernoulli and binomial models for independent trials and the hypergeometric model for sampling a fixed batch, and it shows which conditions decide between them.

Key terms

Bernoulli distribution

Definition A Bernoulli trial is a single random experiment whose outcome falls in exactly one of two categories, called success and failure. Its indicator $X$ takes the value $1$ for a success and $0$ for a failure. If $P(X = 1) = p$ for a number $p$ with $0 \le p \le 1$ , then $X$ has the Bernoulli distribution with success probability $p$ , written $X \sim \operatorname{Bernoulli}(p)$ , and

\begin{gathered} P(X = 1) = p, \\[0.4em] P(X = 0) = 1 - p. \end{gathered}

A variable that takes only the values $0$ and $1$ satisfies $X^2 = X$ , so its second moment is $E(X^2) = p$ . Subtracting the squared mean $p^2$ gives

\begin{gathered} E(X) = p, \\[0.4em] \operatorname{Var}(X) = p(1 - p). \end{gathered}

In plain words Success is simply the outcome being counted, and it can be an unwelcome one such as a defective part or a missed deadline. The mean of a variable that takes only the values $0$ and $1$ is the probability of a $1$ . The variance is $0$ when $p$ is $0$ or $1$ , because the outcome is then certain, and it is largest, equal to $\tfrac14$ , when $p = \tfrac12$ .

Example Inspect one item and record $X = 1$ if it is flagged, where an item is flagged with probability $p = 0.2$ . Then $E(X) = 0.2$ , $\operatorname{Var}(X) = 0.2 \cdot 0.8 = 0.16$ and $\operatorname{SD}(X) = 0.4$ . The mean $0.2$ is not a value that $X$ can take. It is the long-run average of $X$ over many independent repetitions, which is the long-run proportion of flagged items.

Binomial setting

Definition A binomial setting is a sequence of $n$ Bernoulli trials, where $n$ is a positive integer fixed in advance, such that the trials are independent and every trial has the same success probability $p$ . The binomial count is the number of successes,

X = X_1 + X_2 + \dots + X_n,

where $X_i$ is the indicator of a success on trial $i$ .

In plain words The conditions describe how the data were produced, so they come from knowing the mechanism, not from the look of the data. A column of yes and no answers does not show that the answers are independent or that they share one success probability. Repeated outcomes from the same person are often related, and a process whose conditions drift does not keep a fixed $p$ . The number of trials also has to be set beforehand. Counting coin tosses until the first head, for example, does not give a binomial count, because the number of tosses is itself random.

Example Four items come off a process that flags each one independently with probability $0.2$ . This is a binomial setting with $n = 4$ and $p = 0.2$ . Now draw four items one at a time without replacement from a batch of ten that contains exactly two flagged items. Each draw on its own is flagged with probability $0.2$ , but the draws are dependent. The second draw is flagged with probability $\tfrac19$ if the first was flagged and $\tfrac29$ if it was not, so the independence condition fails and this is not a binomial setting.

Binomial distribution

Definition In a binomial setting with $n$ trials and success probability $p$ , the count $X$ of successes has the binomial distribution, written $X \sim \operatorname{Binomial}(n, p)$ , with

\begin{aligned} &P(X = k) \\[0.4em] &\quad = \binom{n}{k} p^k (1 - p)^{n - k} \end{aligned}

for $k = 0, 1, \dots, n$ . By independence, any particular sequence of trials with $k$ successes and $n - k$ failures has probability $p^k (1 - p)^{n - k}$ . There are $\binom{n}{k}$ such sequences, one for each choice of the positions of the successes, and they are disjoint events, so their probabilities add. When $0 < p < 1$ every integer from $0$ to $n$ has positive probability. When $p = 0$ the count is $0$ with probability $1$ , when $p = 1$ it is $n$ with probability $1$ , and the formula still holds in both cases with the convention $0^0 = 1$ .

In plain words The binomial coefficient counts the ways to place the successes among the trials, and the powers give the probability of each placement. Every placement with the same number of successes has the same probability, which is why one product covers them all.

Example For $X \sim \operatorname{Binomial}(4, 0.2)$ , one sequence with two successes, such as success, success, failure, failure, has probability $0.2^2 \cdot 0.8^2 = 0.0256$ , and there are $\binom{4}{2} = 6$ such sequences, so

\begin{aligned} P(X = 2) &= 6 \cdot 0.04 \cdot 0.64 \\[0.4em] &= 0.1536. \end{aligned}

The whole distribution is below, and its masses add to $1$ .

$k$	$0$	$1$	$2$	$3$	$4$
$P(X = k)$	$0.4096$	$0.4096$	$0.1536$	$0.0256$	$0.0016$

Binomial mean and variance

Definition If $X \sim \operatorname{Binomial}(n, p)$ , then

\begin{gathered} E(X) = np, \\[0.4em] \operatorname{Var}(X) = np(1 - p), \\[0.4em] \operatorname{SD}(X) = \sqrt{np(1 - p)}. \end{gathered}

Write the count as $X = X_1 + \dots + X_n$ with each $X_i \sim \operatorname{Bernoulli}(p)$ . Linearity of expectation gives the mean as a sum of $n$ copies of $p$ , and this step needs no independence. The variance of a sum of independent variables is the sum of their variances, which gives $n$ copies of $p(1 - p)$ .

In plain words On average the count equals the number of trials times the success probability. The mean need not be a possible count, and a mean below $1$ does not keep the count below $1$ . For a fixed $n$ the variance is largest when $p = \tfrac12$ , and it is $0$ when $p$ is $0$ or $1$ .

Example For $\operatorname{Binomial}(4, 0.2)$ the mean is $4 \cdot 0.2 = 0.8$ and the variance is $4 \cdot 0.2 \cdot 0.8 = 0.64$ , so the SD is $0.8$ . The table above confirms both. The sum of $k \, P(X = k)$ over all $k$ is $0.8$ , the sum of $k^2 P(X = k)$ is $1.28$ , and $1.28 - 0.8^2 = 0.64$ . Although the mean is below $1$ , two or more flagged items occur with probability $0.1808$ .

Hypergeometric distribution

Definition A population of $N \ge 1$ items contains $K$ marked items, where $0 \le K \le N$ . A sample of $n$ items, with $0 \le n \le N$ , is drawn without replacement so that every subset of $n$ items has the same chance $1 / \binom{N}{n}$ . The number $H$ of marked items in the sample has the hypergeometric distribution with parameters $N$ , $K$ and $n$ , and

P(H = k) = \frac{\binom{K}{k}\binom{N - K}{n - k}}{\binom{N}{n}}

for every integer $k$ with $\max(0, n - (N - K)) \le k \le \min(n, K)$ , while $P(H = k) = 0$ for every other $k$ . The numerator counts the samples made of $k$ of the marked items and $n - k$ of the unmarked ones. The mean is $E(H) = nK/N$ , and with $p = K/N$ and $N \ge 2$ the variance is

\begin{aligned} \operatorname{Var}(H) &= np(1 - p) \\[0.4em] &\quad \times \frac{N - n}{N - 1}. \end{aligned}

In plain words Removing an item changes what is left, so the draws are dependent whenever the batch holds both marked and unmarked items, and only counts that the batch can actually supply are possible. The sample cannot hold more marked items than the batch has or than its own size allows, and a sample larger than the number of unmarked items must contain some marked ones. The factor $(N - n)/(N - 1)$ equals $1$ when $n = 1$ , lies below $1$ when $n \ge 2$ , and equals $0$ when $n = N$ , since a sample of the whole batch gives $H = K$ for certain. This model treats the batch as fixed, and its only randomness is which items get drawn.

Example Draw four items from a batch of ten that contains two flagged items, so $N = 10$ , $K = 2$ and $n = 4$ . The support runs from $\max(0, 4 - 8) = 0$ to $\min(4, 2) = 2$ . Exactly two flagged items means both flagged items and two of the eight others, so

\begin{aligned} P(H = 2) &= \frac{\binom{2}{2}\binom{8}{2}}{\binom{10}{4}} \\[0.4em] &= \frac{28}{210} = \frac{2}{15}. \end{aligned}

The table compares this batch with the independent process of the earlier cards.

Flagged count	$0$	$1$	$2$	$3$	$4$
Binomial	$0.4096$	$0.4096$	$0.1536$	$0.0256$	$0.0016$
Hypergeometric	$\tfrac13$	$\tfrac{8}{15}$	$\tfrac{2}{15}$	$0$	$0$

Both models have mean $0.8$ , and the hypergeometric variance is $0.64 \cdot \tfrac69 = \tfrac{32}{75} \approx 0.4267$ , smaller than the binomial $0.64$ . The batch view and the process view answer different questions. If the ten items were not a known batch but ten outputs of a process that flags each item independently with probability $0.2$ , and the four were sampled without looking at the flags, the count in the sample would again follow $\operatorname{Binomial}(4, 0.2)$ .

Check yourself

A process flags each item independently with probability $0.1$ , and five items are inspected. Find the probability that exactly one is flagged, and the mean and variance of the number flagged.

Show the answer

The five inspections form a binomial setting, so the count is $X \sim \operatorname{Binomial}(5, 0.1)$ and

\begin{aligned} P(X = 1) &= 5 \cdot 0.1 \cdot 0.9^4 \\[0.4em] &= 0.32805. \end{aligned}

The mean is $5 \cdot 0.1 = 0.5$ and the variance is $5 \cdot 0.1 \cdot 0.9 = 0.45$ .

Decide whether a binomial model is justified for each count. First, the number of sixes in 12 rolls of a fair die. Second, the number of days in one week on which a particular commuter arrives late. Third, the number of rolls of a fair die needed to get the first six.

Show the answer

The first count is binomial. The number of rolls is fixed at 12, the rolls are independent and each gives a six with probability $\tfrac16$ , so the count is $\operatorname{Binomial}(12, \tfrac16)$ with mean $2$ .

The second count is made of yes and no outcomes, but nothing guarantees that the days are independent or share one probability of lateness. Late days can cluster, for example during a week of roadwork, and Mondays may differ from Fridays. A binomial model would need evidence for both conditions.

The third count is not binomial. The number of rolls is the random quantity being recorded, so there is no fixed number of trials whose successes are counted.

A club roster lists 12 members, 4 of whom are volunteers. Three members are chosen without replacement, with every set of three equally likely. An analyst uses $\operatorname{Binomial}(3, \tfrac13)$ and reports the probability of choosing no volunteers as $(\tfrac23)^3 = \tfrac{8}{27} \approx 0.296$ . Which condition fails, and what is the correct probability?

Show the answer

The draws are not independent. If the first member chosen is not a volunteer, the second is a volunteer with probability $\tfrac{4}{11}$ , not $\tfrac{4}{12}$ . The count is hypergeometric with $N = 12$ , $K = 4$ and $n = 3$ . No volunteers means three of the eight other members, so

\begin{aligned} P(H = 0) &= \frac{\binom{8}{3}}{\binom{12}{3}} = \frac{56}{220} \\[0.4em] &= \frac{14}{55} \approx 0.2545. \end{aligned}

Both models give a mean of $1$ volunteer, but the hypergeometric variance is $3 \cdot \tfrac13 \cdot \tfrac23 \cdot \tfrac{9}{11} = \tfrac{6}{11} \approx 0.545$ , smaller than the binomial $\tfrac23$ .

Bernoulli, Binomial and Hypergeometric Models

Key terms

Bernoulli distribution

Binomial setting

Binomial distribution

Binomial mean and variance

Hypergeometric distribution

Check yourself

How this lesson connects

Builds on

Used later in