The Binomial Distribution, introduction to probability lesson

Five independent checks each succeed with probability $0.4$ . How likely is it that exactly two of them succeed, and how likely is it that at least one does? The probability of one particular sequence of results comes from multiplying, but many different sequences give the same number of successes. The binomial distribution collects those sequences into one formula. This lesson derives it, finds the mean, the variance and the tail probabilities, and states the conditions the model needs.

Key terms

Binomial assumptions

Definition A count $X$ of successes has a binomial distribution when the experiment meets four conditions.

The number of trials $n$ is fixed in advance and does not depend on the results.
Each trial records whether one clearly defined success event occurs.
The trials are independent.
Every trial has the same success probability $p$ .

Under these conditions $X \sim \operatorname{Bin}(n, p)$ . When a condition fails, the count is still a random variable with its own distribution, but the binomial formula is no longer justified for it.

In plain words The formula needs a fixed list of trials that do not influence each other and that all have the same chance of success. A rule that stops after a chosen number of successes breaks the first condition. Draws without replacement break independence whenever the collection holds both kinds of items, because each draw changes what is left. Trials with different success probabilities break the fourth condition.

Example A bag holds $4$ marked tokens among $10$ . Drawing $3$ tokens with replacement gives a number of marked draws with distribution $\operatorname{Bin}(3, 2/5)$ . Without replacement, a marked first draw lowers the chance of a marked second draw to $\tfrac{3}{9}$ , so the draws are dependent and the binomial assumptions fail. For this bag every count from $0$ to $3$ has a probability different from its $\operatorname{Bin}(3, 2/5)$ value. Unequal probabilities matter too. Two independent trials with success probabilities $0.2$ and $0.6$ give two successes with probability $0.2 \cdot 0.6 = 0.12$ , while $\operatorname{Bin}(2, 0.4)$ , which uses their average, gives $0.4^2 = 0.16$ .

Binomial distribution

Definition Let $n$ be a positive integer and $0 \le p \le 1$ . The binomial distribution $\operatorname{Bin}(n, p)$ is the distribution of the number of successes in $n$ trials that satisfy the binomial assumptions. If $X$ is that count and $I_j$ is the indicator of success on trial $j$ , then

X = I_1 + I_2 + \cdots + I_n,

where the indicators are independent and each is $\operatorname{Bernoulli}(p)$ . The count takes values in $\{0, 1, \dots, n\}$ , and each of these values has positive probability when $0 < p < 1$ . The case $n = 1$ is the Bernoulli distribution. At the endpoints, $X = 0$ with probability $1$ when $p = 0$ , and $X = n$ with probability $1$ when $p = 1$ .

In plain words The count keeps the number of successes and forgets where they happened. Different sequences of results can lead to the same count, and the binomial distribution gathers all the sequences with a given count.

Example Five independent checks each succeed with probability $0.4$ , so the number of successes is $X \sim \operatorname{Bin}(5, 0.4)$ with possible values $0$ to $5$ . Writing S for success and F for failure, the sequences SSFFF and FSFSF are different results, but both give $X = 2$ .

Binomial probability formula

Definition If $X \sim \operatorname{Bin}(n, p)$ , then for each integer $k$ with $0 \le k \le n$ the PMF is

p_X(k) = \binom{n}{k} p^{k} (1 - p)^{n - k},

and $p_X(k) = P(X = k) = 0$ for every other value of $k$ . By independence and the common $p$ , each particular sequence with $k$ successes and $n - k$ failures has probability $p^{k}(1 - p)^{n - k}$ . There are $\binom{n}{k}$ such sequences, one for each choice of the $k$ success positions, and they are mutually exclusive, so their probabilities add. The masses sum to $1$ by the binomial theorem, since

\begin{aligned} &\sum_{k=0}^{n} \binom{n}{k} p^{k} (1 - p)^{n - k} \\[0.4em] &\quad = \bigl(p + (1 - p)\bigr)^{n} = 1. \end{aligned}

At $p = 0$ or $p = 1$ the formula relies on the convention $0^0 = 1$ .

In plain words The coefficient counts the arrangements, the power of $p$ accounts for the successes in one arrangement, and the power of $1 - p$ accounts for its failures. Leaving out the coefficient gives the probability of a single arrangement. Leaving out the failure factor ignores what the other trials must do.

Example For $X \sim \operatorname{Bin}(5, 0.4)$ , the single sequence SSFFF has probability $\bigl(\tfrac{2}{5}\bigr)^2 \bigl(\tfrac{3}{5}\bigr)^3 = \tfrac{108}{3125} = 0.03456$ . Each of the $\binom{5}{2} = 10$ sequences with two successes has that probability, so

\begin{aligned} P(X = 2) &= 10 \cdot \frac{108}{3125} \\[0.4em] &= \frac{216}{625} = 0.3456. \end{aligned}

The whole PMF is below. These decimals are exact, and they add to $1$ .

$k$	$P(X = k)$
$0$	$0.07776$
$1$	$0.2592$
$2$	$0.3456$
$3$	$0.2304$
$4$	$0.0768$
$5$	$0.01024$

Mean of a binomial count

Definition If $X \sim \operatorname{Bin}(n, p)$ , then

E[X] = np.

Writing $X = I_1 + \cdots + I_n$ with $E[I_j] = p$ for each $j$ , linearity of expectation gives $E[X] = p + \cdots + p = np$ . This step uses only linearity, not independence, so any sum of $n$ indicators that each equal $1$ with probability $p$ has mean $np$ , even when the sum is not binomial.

In plain words On average a fraction $p$ of the trials succeed. The mean is the balance point of the distribution, not a promise about any one group of trials, and it need not be a possible value of the count.

Example For $\operatorname{Bin}(5, 0.4)$ the mean is $5 \cdot 0.4 = 2$ , yet $P(X = 2) = 0.3456$ , so most of the probability, $0.6544$ , lies on other counts. For $\operatorname{Bin}(4, 0.3)$ the mean is $1.2$ , which is not a possible count. The number of marked tokens in three draws without replacement from the bag with $4$ marked tokens among $10$ also has mean $3 \cdot \tfrac{2}{5} = \tfrac{6}{5}$ , because each draw is marked with probability $\tfrac{2}{5}$ before anything is seen, even though that count is not binomial.

Variance of a binomial count

Definition If $X \sim \operatorname{Bin}(n, p)$ , then

\operatorname{Var}(X) = np(1 - p),

and $\operatorname{SD}(X) = \sqrt{np(1 - p)}$ . Because the indicators $I_1, \dots, I_n$ are independent, the variance of their sum is the sum of their variances, each equal to $p(1 - p)$ . This step uses independence. For dependent indicators the variance of the count can differ from $np(1 - p)$ even when the mean is still $np$ . Both endpoints $p = 0$ and $p = 1$ give variance $0$ .

In plain words For a fixed $p$ strictly between $0$ and $1$ , the spread of the count grows with the number of trials, but the standard deviation grows like $\sqrt{n}$ , not like $n$ . The standard deviation of the count is the square root of the summed variances, not the sum of the standard deviations of the trials.

Example For $\operatorname{Bin}(5, 0.4)$ the variance is $5 \cdot 0.4 \cdot 0.6 = \tfrac{6}{5} = 1.2$ , and $\operatorname{SD}(X) = \sqrt{1.2} \approx 1.10$ . Adding the five trial standard deviations would give $5\sqrt{0.24} \approx 2.45$ , which is wrong. For three draws without replacement from the bag with $4$ marked tokens among $10$ , the count of marked tokens has mean $1.2$ , the same as $\operatorname{Bin}(3, 0.4)$ , but its variance is $0.56$ rather than $3 \cdot 0.4 \cdot 0.6 = 0.72$ .

Binomial tail probability

Definition A tail probability of $X \sim \operatorname{Bin}(n, p)$ is the probability that the count lies at or beyond a threshold, such as $P(X \ge k)$ or $P(X \le k)$ . It is a finite sum of PMF values. For each integer $k$ with $1 \le k \le n$ ,

P(X \ge k) = \sum_{j=k}^{n} P(X = j),

and the complement rule gives the same number as $1 - P(X \le k - 1)$ . The phrase “at least $k$ ” includes $k$ , while “more than $k$ ” starts at $k + 1$ .

In plain words First translate the words into a list of integer counts. Then add the matching PMF values, or subtract the excluded ones from $1$ when that list is shorter.

Example For $\operatorname{Bin}(5, 0.4)$ , at least one success excludes only the count $0$ , which requires all five trials to fail. So

\begin{aligned} P(X \ge 1) &= 1 - \Bigl(\frac{3}{5}\Bigr)^5 \\[0.4em] &= \frac{2882}{3125} \\[0.4em] &= 0.92224. \end{aligned}

At least three successes is the set of counts $\{3, 4, 5\}$ , so $P(X \ge 3) = 0.2304 + 0.0768 + 0.01024 = 0.31744$ . A high chance of at least one success does not make each trial likely to succeed, since each one succeeds with probability only $0.4$ .

Check yourself

Four independent trials each succeed with probability $\tfrac{1}{4}$ , and $X$ counts the successes. Find $P(X = 1)$ , $E[X]$ and $\operatorname{Var}(X)$ .

Show the answer

The count is $\operatorname{Bin}(4, 1/4)$ . There are $\binom{4}{1} = 4$ positions for the single success, so $P(X = 1) = 4 \cdot \tfrac{1}{4} \cdot \bigl(\tfrac{3}{4}\bigr)^3 = \tfrac{27}{64}$ . The mean is $4 \cdot \tfrac{1}{4} = 1$ , and the variance is $4 \cdot \tfrac{1}{4} \cdot \tfrac{3}{4} = \tfrac{3}{4}$ .

For the same model, find $P(X \ge 2)$ . A classmate computes $1 - P(X = 0) - P(X = 1) - P(X = 2)$ instead. Which probability did the classmate find?

Show the answer

At least two successes is the set of counts $\{2, 3, 4\}$ , so the complement removes only $0$ and $1$ . The probabilities are $P(X = 0) = \bigl(\tfrac{3}{4}\bigr)^4 = \tfrac{81}{256}$ and $P(X = 1) = \tfrac{108}{256}$ , so

\begin{aligned} P(X \ge 2) &= 1 - \frac{189}{256} \\[0.4em] &= \frac{67}{256} \approx 0.262. \end{aligned}

The classmate also removed the count $2$ , which belongs to the event, and so found $P(X \ge 3) = \tfrac{13}{256}$ .

A box holds $10$ parts, $4$ of them defective. An inspector draws $3$ parts without replacement and counts the defective ones. Is the count $\operatorname{Bin}(3, 2/5)$ ? Compare the probability that all three are defective with the binomial value.

Show the answer

No. The number of draws is fixed, and each draw is defective with probability $\tfrac{2}{5}$ before anything is seen, but the draws are dependent. After a defective first draw, the second is defective with probability $\tfrac{3}{9} = \tfrac{1}{3}$ , not $\tfrac{2}{5}$ . The exact probability that all three are defective is

\frac{4}{10} \cdot \frac{3}{9} \cdot \frac{2}{8} = \frac{1}{30} \approx 0.033,

while $\operatorname{Bin}(3, 2/5)$ gives $\bigl(\tfrac{2}{5}\bigr)^3 = \tfrac{8}{125} = 0.064$ . With replacement the draws would be independent with the same success probability, and the binomial model would be exact.

The Binomial Distribution

Key terms

Binomial assumptions

Binomial distribution

Binomial probability formula

Mean of a binomial count

Variance of a binomial count

Binomial tail probability

Check yourself

How this lesson connects

Builds on

Used later in

Where this leads in Statistics