The Addition Rule, introduction to probability lesson

One of 30 numbered cards is drawn, each card equally likely. Twelve cards carry a blue mark, fifteen carry a star and five carry both. What is the probability that the drawn card is blue or starred? Adding $12$ and $15$ gives $27$ , but the five cards with both marks have been counted twice. This lesson derives the addition rule from the disjoint regions of two events, uses the same regions for exactly one and neither, and bounds the probability of a union when the overlap is not known.

Key terms

Overlap

Definition The overlap of two events $A$ and $B$ is their intersection $A \cap B$ , the set of outcomes that belong to both. The events are disjoint, or mutually exclusive, when $A \cap B = \varnothing$ . The sum $P(A) + P(B)$ includes the probability of the overlap twice, once inside each event.

In plain words Two descriptions can fit the same outcome without making it two outcomes. A card that is both blue and starred appears on both lists, but it is one card, so its probability belongs in the union once.

Example Number the cards 1 through 30, each drawn with probability $1/30$ . Let $A = \{1, 2, \dots, 12\}$ be the blue cards and $B = \{8, 9, \dots, 22\}$ the starred cards. The overlap is $A \cap B = \{8, 9, 10, 11, 12\}$ , with five cards. The total $12 + 15 = 27$ counts these five cards twice, and the union has $22$ cards.

Disjoint regions

Definition Two events $A$ and $B$ split $\Omega$ into four pairwise disjoint regions,

\begin{gathered} A \setminus B, \quad A \cap B, \\[0.4em] B \setminus A, \quad (A \cup B)^{c}, \end{gathered}

the outcomes in $A$ only, in both, in $B$ only and in neither. Their union is $\Omega$ , so their probabilities add to $1$ . Each event is the union of two regions, so $P(A) = P(A \setminus B) + P(A \cap B)$ and $P(B) = P(B \setminus A) + P(A \cap B)$ .

In plain words A Venn diagram of two events has four pieces, and every outcome lies in exactly one of them. A number written for a whole circle is an event total, the sum of the piece outside the other circle and the overlap.

Example For the cards, the region counts are $12 - 5 = 7$ blue only, $5$ with both marks, $15 - 5 = 10$ starred only and $8$ with no mark. They add to $7 + 5 + 10 + 8 = 30$ . The numbers $12$ and $15$ are event totals, not region counts.

Addition rule

Definition For any two events $A$ and $B$ of a probability model,

\begin{aligned} P(A \cup B) &= P(A) + P(B) \\[0.4em] &\quad - P(A \cap B). \end{aligned}

For the proof, the union is the disjoint union of $A \setminus B$ , $A \cap B$ and $B \setminus A$ , so additivity gives

\begin{aligned} P(A \cup B) &= P(A \setminus B) \\[0.4em] &\quad + P(A \cap B) \\[0.4em] &\quad + P(B \setminus A). \end{aligned}

Substituting $P(A \setminus B) = P(A) - P(A \cap B)$ and $P(B \setminus A) = P(B) - P(A \cap B)$ gives the rule. For disjoint events $P(A \cap B) = P(\varnothing) = 0$ , and the rule reduces to $P(A \cup B) = P(A) + P(B)$ . The rule assumes neither equally likely outcomes nor independence.

In plain words Add the two event probabilities, then subtract the overlap once, because the sum counted it twice. The outcomes in the overlap are still part of the union, and the subtraction removes only the second copy of their probability.

Example With equally likely cards every probability is a count divided by $30$ , so

\begin{aligned} P(A \cup B) &= \frac{12}{30} + \frac{15}{30} - \frac{5}{30} \\[0.4em] &= \frac{22}{30} = \frac{11}{15}. \end{aligned}

Now move the stars but keep $P(A) = 0.4$ and $P(B) = 0.5$ . If the stars sit on cards 13 to 27, no card has both marks, and $P(A \cup B) = 27/30 = 9/10$ . If the stars sit on cards 1 to 15, every blue card is starred, so $A \subseteq B$ , the overlap is $A$ itself and $P(A \cup B) = P(B) = 1/2$ . These two event probabilities alone do not determine the union. The overlap, or enough information to find it, is needed.

Exactly one of two events

Definition The event that exactly one of $A$ and $B$ occurs is $(A \setminus B) \cup (B \setminus A)$ , also written $A \triangle B$ . Its probability is

\begin{aligned} P(A \triangle B) &= P(A) + P(B) \\[0.4em] &\quad - 2P(A \cap B). \end{aligned}

The event that neither occurs is $(A \cup B)^{c} = A^{c} \cap B^{c}$ , and the complement rule gives it probability $1 - P(A \cup B)$ .

In plain words In probability “A or B” is the inclusive or, meaning $A$ or $B$ or both. “Exactly one” leaves out the outcomes in both, so the overlap is subtracted twice, once for the duplicate count and once more to remove it altogether. “Neither” is everything outside the union.

Example For the cards, exactly one mark covers the $7$ blue-only and the $10$ starred-only cards, so

\begin{aligned} P(A \triangle B) &= \frac{12 + 15 - 2 \cdot 5}{30} \\[0.4em] &= \frac{17}{30}. \end{aligned}

Neither covers the $8$ unmarked cards, so its probability is $8/30 = 4/15$ , which agrees with $1 - 11/15$ . The events ” $A$ or $B$ ”, “exactly one” and “neither” are different events with different probabilities, $22/30$ , $17/30$ and $8/30$ .

Union bound

Definition For any two events $A$ and $B$ ,

\begin{aligned} P(A \cup B) &\le P(A) + P(B), \\[0.4em] P(A \cup B) &\ge P(A), \\[0.4em] P(A \cup B) &\ge P(B). \end{aligned}

The first inequality is the union bound. It follows from the addition rule because $P(A \cap B) \ge 0$ , and it is an equality exactly when $P(A \cap B) = 0$ . The other two follow from monotonicity, since $A$ and $B$ are contained in $A \cup B$ . Together with $P(A \cup B) \le 1$ , they place the union between the larger of $P(A)$ and $P(B)$ and the smaller of $1$ and $P(A) + P(B)$ . Applying the two-event bound repeatedly gives the union bound for finitely many events,

P\Bigl(\bigcup_{i=1}^{n} A_i\Bigr) \le \sum_{i=1}^{n} P(A_i).

In plain words When the overlap is unknown, the two event probabilities still bracket the probability of the union. The union is at least as likely as the more likely of the two events, and at most as likely as the two probabilities added, or $1$ if that sum is larger. No independence is assumed. The sum $P(A) + P(B)$ is a ceiling that holds in every model, and it equals the probability of the union exactly when the overlap has probability $0$ .

Example For the cards $P(A) = 0.4$ and $P(B) = 0.5$ , so any model with these two probabilities has $0.5 \le P(A \cup B) \le 0.9$ . The original answer $11/15 \approx 0.733$ lies inside this range. The changed placements of the stars reach both ends, $1/2$ when blue is contained in starred and $9/10$ when the two events are disjoint. An answer below $0.5$ or above $0.9$ would signal a mistake, but a value inside the range still needs the overlap to be confirmed.

Check yourself

One of the cards numbered 1 through 30 is drawn, each equally likely. Let $C$ be the event that the number is divisible by $3$ and $D$ the event that it is greater than $20$ . Find $C \cap D$ and the probability of $C \cup D$ .

Show the answer

The event $C = \{3, 6, 9, \dots, 30\}$ has $10$ numbers and $D = \{21, 22, \dots, 30\}$ has $10$ . The overlap is $C \cap D = \{21, 24, 27, 30\}$ , with $4$ numbers. The addition rule gives

\begin{aligned} P(C \cup D) &= \frac{10 + 10 - 4}{30} \\[0.4em] &= \frac{16}{30} = \frac{8}{15}. \end{aligned}

The answer passes the bounds check, since it lies between $1/3$ and $2/3$ .

For the same events, find the probability that exactly one of $C$ and $D$ occurs and the probability that neither occurs. A classmate answers $16/30$ for exactly one. What went wrong?

Show the answer

Exactly one removes the overlap twice,

\begin{aligned} P(C \triangle D) &= \frac{10 + 10 - 2 \cdot 4}{30} \\[0.4em] &= \frac{12}{30} = \frac25. \end{aligned}

The twelve numbers are $3, 6, 9, 12, 15, 18$ in $C$ only and $22, 23, 25, 26, 28, 29$ in $D$ only. Neither has the remaining $30 - 16 = 14$ numbers, so its probability is $14/30 = 7/15$ , and $8/15 + 7/15 = 1$ as it should. The classmate computed the inclusive or, which also counts the four numbers that are in both events.

Two events have $P(A) = 0.7$ and $P(B) = 0.6$ . Can they be disjoint? Which values can $P(A \cup B)$ and $P(A \cap B)$ take?

Show the answer

They cannot be disjoint, because disjoint events would give $P(A \cup B) = 0.7 + 0.6 = 1.3$ , which is more than $1$ . The bounds give $0.7 \le P(A \cup B) \le 1$ . The addition rule says $P(A \cap B) = 1.3 - P(A \cup B)$ , so the overlap lies between $0.3$ and $0.6$ .

Every value in these ranges is possible. For any $x$ with $0.3 \le x \le 0.6$ , give the four regions the probabilities $x$ for both, $0.7 - x$ for $A$ only, $0.6 - x$ for $B$ only and $x - 0.3$ for neither. These are nonnegative and add to $1$ , so they define a valid model with $P(A \cap B) = x$ and $P(A \cup B) = 1.3 - x$ .

The Addition Rule

Key terms

Overlap

Disjoint regions

Addition rule

Exactly one of two events

Union bound

Check yourself

How this lesson connects

Builds on

Used later in